Share via

Debug Fundamentals Exercise 1: Reverse engineer a function

  • Article

 

Hello ntdebuggers! We’ve seen a lot of interest in our Puzzlers, and we’ve also seen requests and interest in topics covering debugging fundamentals. So we’ve decided to combine the two topics and post a series of “Fundamentals Exercises”. These exercises will be designed more as learning experiences rather than simply puzzlers. We hope you find them interesting and educational!

 

Feel free to post your responses here, but we won’t put them on the site until after we post the “official” responses, so as to avoid spoilers.

 

 

Examine the following code, registers, and stack values to determine the following:

 

1. When the function “DoTheWork” returns, what is the return value from that function?

2. Bonus: what is the mathematical operation that “DoTheWork” performs?

 

Hints:

1. The bracket notation [] in the assembly means to treat the value in brackets as a memory address, and access the value at that address.

2. 32-bit integer return values are stored in eax

 

// Code

0:000> uf eip

demo2!DoTheWork:

0040101c 55 push ebp

0040101d 8bec mov ebp,esp

0040101f 8b4d08 mov ecx,dword ptr [ebp+8]

00401022 8bc1 mov eax,ecx

00401024 49 dec ecx

00401025 0fafc1 imul eax,ecx

00401028 83f902 cmp ecx,2

0040102b 7ff7 jg demo2!DoTheWork+0x8 (00401024)

0040102d 5d pop ebp

0040102e c3 ret

// Current register state

0:000> r

eax=00000007 ebx=7ffd9000 ecx=ffffffff edx=00000007 esi=00001771 edi=00000000

eip=0040101c esp=0012fe9c ebp=0012feac iopl=0 nv up ei pl nz na po nc

cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202

demo2!DoTheWork:

0040101c 55 push ebp

// Current stack values for this thread

0:000> dps esp

0012fe9c 00406717 demo2!main+0x27

0012fea0 00000007

0012fea4 82059a87

0012fea8 00000007

0012feac 0012ff88

0012feb0 004012b2 demo2!mainCRTStartup+0x170

0012feb4 00000002

0012feb8 00980e48

0012febc 00980e80

0012fec0 00000094

0012fec4 00000006

0012fec8 00000000

0012fecc 00001771

0012fed0 00000002

0012fed4 76726553

0012fed8 20656369

0012fedc 6b636150

0012fee0 00003120

0012fee4 00000000

0012fee8 00000000

0012feec 00000000

0012fef0 00000000

0012fef4 00000000

0012fef8 00000000

0012fefc 00000000

0012ff00 00000000

0012ff04 00000000

0012ff08 00000000

0012ff0c 00000000

0012ff10 00000000

0012ff14 00000000

0012ff18 00000000

 

[Update: our answer. Posted 11/19/2008]

Wow - what a great response from our readers on this exercise! It is great to see the various approaches to reverse engineering this code. As for the answer, the numerical result (stored in eax) is 5040, and the corresponding mathematical operation is a factorial. So 7! is the result calculated, given that 7 was passed to the function. Congratulations to all of you that got it right!

Many of you posted some C code to give an idea of what the original source of DoTheWork() might have looked like. The original source was actually written in assembly! However, it was written to be called from C, and it uses ebp in the same way that compiled C code might. This function wasn’t written with optimal performance in mind, but rather for learning about reverse engineering.

Comments

  • Anonymous
    November 13, 2008
    I'm betting on the factorial function .

  • Anonymous
    November 13, 2008
    DoTheWork performs the factorial operation the stack says we're in DoTheWork(7) so the return value is 7! = 5040 nice puzzlers, keep it up !

  • Anonymous
    November 13, 2008
    DoTheWork is the factorial function; as such it'll return 7!.

  • Anonymous
    November 13, 2008
    [ebp] -> Old Ebp == 0012ff88 [ebp+4] -> return address == 004012b2 [ebp+8] -> Param 1 == 2 In this case: 2 * (2 - 1) = 2 int DoTheWork(int n) { int i = n; do {  n--;  i = i * n; } while (n > 2); return (i); } BTW, why not make this a fastcall :)?

  • Anonymous
    November 13, 2008
    (1) At function exit, EAX = 0x13b0 (2) The function computes the factorial of its argument.

  • Anonymous
    November 13, 2008
    Looks like factorial, assuming the input is positive (doesn't handle 0!)... result will be 7! = 5040. <code> int DoTheWork(int x) { int r = x; do { r *= --x; } while (x > 2); return r; } </code>

  • Anonymous
    November 13, 2008

  1. Return value should be 5040 (in decimal)
  2. Looks like the !n function called with n=7

  • Anonymous
    November 13, 2008
    Doh! didn't look at esp 1.) 7! 2.) Factorial

  • Anonymous
    November 13, 2008
    I'm going to refrain from posting my answer since I don't want to spoil the challenge for anyone.  But this was a really fun post (to me, at least), so thanks for it!

  • Anonymous
    November 13, 2008
    This might be completly wrong but off the bat I would translate it to something like this: void* DoTheWork(void* e, int p) {  do  {    p = ((--p) * p;  }  while (p != 2); return e; } How close did i got it? :=)

  • Anonymous
    November 13, 2008
    I guess this is this: http://en.wikipedia.org/wiki/Factorial int DoTheWork(int p1) { int eax; int ecx;    ecx = p1;    eax = p1;    do    {        ecx -= 1;        eax *= ecx;    } while (ecx > 2);    return eax; } 0012feb4  00000002 Here is the argument you used. The function returns 2.

  • Anonymous
    November 13, 2008
    It calculates the number 7 factorial. The return value will be 7! = 5040. -George

  • Anonymous
    November 13, 2008
    Great idea with an exercise! The function calculates the factorial of the argument. The result in the example is 7!. int fact(int arg1) {    int retval = arg1;    do {        retval *= --arg1;    } while (arg1 > 2);    return retval; }

  • Anonymous
    November 14, 2008
    I assume the result is 5040. About the function I am not quiet sure.

  • Anonymous
    November 14, 2008
    I assume the result is 5040. About the math function I am not sure.

  • Anonymous
    November 14, 2008
    <pre> #define DoTheWork factorial int DoTheWork(int x) { int y=x-1; for(;y>=2;--y) x*=y; return x; } DoTheWork(7)=5040 </pre>

  • Anonymous
    November 14, 2008
    Looks like you're calculating 7! which would be 5040.

  • Anonymous
    November 14, 2008
    The comment has been removed

  • Anonymous
    November 14, 2008

  1.       When the function “DoTheWork” returns, what is the return value from that function? 5040
  2.       Bonus: what is the mathematical operation that “DoTheWork” performs? factorial

  • Anonymous
    November 14, 2008
    Here the answer. int DoWork(int x) { int y = x; do { y *= --x; }    while(x > 2); return y; } "DoWork" calculates factorial.

  • Anonymous
    November 14, 2008
    Hello, here is the answer: int DoWork(int x) { int y = x; do { y *= --x; }        while(x > 2); return y; } Function calculates factorial for its argument.

  • Anonymous
    November 14, 2008

  1. It returns 7!, ie 5040
  2. It computes the factorial of the input number

  • Anonymous
    November 14, 2008
    I am new to assembly debugging, but let me try this. It seem this code is calculating the factorial of a number and then store the result in EAX. The code decrements ECX for each loop until it becomes 2. and each time the value in ECX will be multiplied into EAX. Here is the part that confuses me. The initial value of EAX is from ECX and ECX got its value from the memory location pointed by ebp+8. and ebp got its inital value from esp which is probably "0012fe9c". So 0012fe9c+8 = 0012fea4. so [0012fea4] is "82059a87". but this seems a very large number already. so the result in EAX will be the factorial of "82059a87"? I could be completely wrong here. Just a wild guess.

  • Anonymous
    November 14, 2008
    It's a factorial function.  The original value is 7 and the result (7!) is 0x13B0 or 5040 decimal.

  • Anonymous
    November 14, 2008
    this seems to be computing the factorial of a number, though I am still confused what the end result is, because I have a hard time finding the initial value of ECX...

  • Anonymous
    November 14, 2008
    Looks like a factorial function, in this case it should return 7!, or 5040. The original code probably looks something like this: DoTheWork(int x) {  int y = x;  do  {    x--;    y = y * x;  }  while (x > 2)  return y; }

  • Anonymous
    November 14, 2008
    This is factorial(7) - it will return 5040.

  • Anonymous
    November 14, 2008
    /// Calculates factorial int DoTheWork(int n) {    int result = n;    do {        result *= --n;    }    while(n>2);    return result; } // Just curious - was the code from puzzle compiled from C++ or hand-written in assembly? Could not get exactly the same assembly from my msvc 2008 - Release build omits frame creation, while Debug build saves too much regs and 0xCC-fills frame.

  • Anonymous
    November 14, 2008

  1. Looks like it is calculating factorial of the number passed as parameter
  2. Return value is 2

  • Anonymous
    November 15, 2008
    I recently started my studies on debugging. So I can be completely wrong. This function seems to be a factorial: Mainly when looking at 00401024 until 00401028 where the function, decrement ecx until it becomes 2, and at each DEC, it multiply EAX * ECX But I am really unable to find the final value because the ECX receive [ebp+8] at 40101f and [ebp+8] seems to be 82059a87. EBP = 12fe9c EBP+8 = 12FEA4 12fea4  = 82059a87 (????) So... I was unable to find factorial for 82059a87... Maybe I get something wrong... Thanks for the opportunity

  • Anonymous
    November 15, 2008

  1. 5040
  2. factorial Please, next time a bit more complicated. ;)
  • Anonymous
    November 15, 2008
    <i>Here's hoping that this looks like it did in plain text...</i> <pre> My Answers:
  1.      5040 (7!)
  2.      Factorial N  (N!) ///////////////////////////////////////////////////////////////////////////// // //  Detailed explanation(s) // ///////////////////////////////////////////////////////////////////////////// ///////////////////////////////////////////////////////////////////////////// //  Function:   demo2!DoTheWork:    (In assembly) ///////////////////////////////////////////////////////////////////////////// // //  Save the Prior Frame Pointer to the stack // 0040101c 55              push    ebp // //  Set the Frame pointer to the current Stack pointer // 0040101d 8bec            mov     ebp,esp // //  Right at this point, the stack looks like: //  EBP = ESP //   //  EPB - N -- Local variables, if any (here there aren't) //  EBP     -- Old EBP //  EBP + 4 -- Return Address back to calling function //  EBP + 8 -- First function Arg // // //  Put Arg1 into ECX // 0040101f 8b4d08          mov     ecx,dword ptr [ebp+8] // //  Copy ECX into EAX // 00401022 8bc1            mov     eax,ecx // //  LOOP:   ECX-- // 00401024 49              dec     ecx // //  EAX = EAX * ECX // 00401025 0fafc1          imul    eax,ecx // //  If EXC is greater than 2, goto LOOP: // 00401028 83f902          cmp     ecx,2 0040102b 7ff7            jg      demo2!DoTheWork+0x8 (00401024) // //  Else it wasn't, so replace the Old Frame Pointer // 0040102d 5d              pop     ebp // //  Return back to the calling function. //  Whatever is in EAX is effectively returned. // 0040102e c3              ret ///////////////////////////////////////////////////////////////////////////// //  Function:   demo2!DoTheWork: (In C) ///////////////////////////////////////////////////////////////////////////// int DoTheWork(int Number) {    int WorkingValue = Number;    int Factorial    = WorkingValue;    do {        WorkingValue--;        Factorial *= WorkingValue;    }while (WorkingValue > 2);    return(Factorial); } // //  Since we know what the function does, all we need to do is find the //  argument to it.  We can just look at the stack... // 0012fe9c  00406717 demo2!main+0x27              // Return address for DoTheWork 0012fea0  00000007                              // Arg1 "7" 0012fea4  82059a87 </pre>

  • Anonymous
    November 15, 2008
    Hello, a source code which produces almost directly the same code is as follows: int DoThework(int value) {   int c = value;   do {     value = value * --c;   } while (c > 2);   return value; } Thus, this function multiplies value * value - 1 * value - 2 * ... * 2, and then ends. Thus, for positive values (value > 0), this is the faculty (value!) function. For value <= 0, the function calculates value * (value-1), as the loop will be executed exactly once.

  • Anonymous
    November 16, 2008
    I think the result is Factorial(2).

  • Anonymous
    November 17, 2008
    Well it seems to be calculating factorial of 82059a87 .. Am I right :) ?

  • Anonymous
    November 17, 2008
    Very good exercise. The return value will be 0x2. And the function is for calculating factorial of a number.

  • Anonymous
    November 19, 2008
    I believe returned value will be 5040, and the code implements a factorial function. I recognized the function as factorial, but when I was about to calculate the return value I first thought that dword ptr [ebp+8] means 0012fea4  82059a87 Then I realized that 0040101c 55              push    ebp has not yet been executed, and it will but another 4bytes on stack. Anyway, I'm not a low-level debugging expert, but I enjoy these a lot. Thanks for the exercise. Hope there will be a post with the full explanation later on.

  • Anonymous
    November 19, 2008

  1. eax holds 5040.
  2. Factorial!