I have an ELF executable I'm working on (got it from a previous CTF competition). The executable simply asks for a password, and then it prints out "congrats".
The code snippets and my annotations are long, so I've included them in separate HTML links.
//Gives you an overview of what this program does in C.
int main(int argc, char ** argv)
{
int32_t result; // success value
if (argc > 1)
{
setup(); //Allocates an integer array (g1)
int32_t v1 = 0; //counter
while (true)
{
char v2 = *(char *) (*(int32_t *) ((int32_t)argv + 4) + v1);
int32_t v3 = *(int32_t *) (4 * v1 + (int32_t) & g1);
int32_t v4 = v1 + 1; //index in g1 array
if (check( (int32_t *) v3, (int32_t) v2) != 0)
{
puts("Nope");
return -1;
}
if (v4 >= 31) //when we've reached our last index in array
{
break;
}
v1 = v4;
}
puts("congrats!");
result = 0;
}
else
{
puts("usage ./smoothie ");
result = -1;
}
return result;
}
I've tried using GDB to debug through the program and IDA to understand the control-flow. My deduction ends up with 3 things:
- setup() generates some sort of integer array
- check(int32_t * array, int * array) gets called 31 times to confirm our "password" as indicated by the possible while loop
- Inside check(...), it goes through each malloc'd array[5] and confirms each "char" of our password.
Here is IDA's control flow graph of check() (simple - 3 parts):

Here is the control flow graph of check() click to enlarge:
The majority of check() compares * (array) for the upper half and * (array + 8) for the bottom half plus the passwordIndex comparison.
In the main(int argc, char ** argv) of this program, there is a while loop that needs to be executed 31 times (the password is possibly 31 characters long). The while loop will exit if check() returns 1.
I think the middle section does some calculations (add, sub, cdq & idiv, imul, and xor) that might be useful towards knowing what to pass as a password.
Please guide me in the right direction (with hints if you can). Debugging the program yourself might make my question more clear.
False Lead
- In check(), I'm assuming that the first set of checks "calculate" the correct number that my input ascii should match (e.g. calculated 90, so index[i] should equal 'Z' ~ 90).
- In the second half of check it uses * (array + 8), so that means that its using the array's 3rd number for the 101, 142, ... , number comparisons. We could change the dereferenced value of * (array + 8) to the calculated number (90) to get to the final comparison.
- Once we reach the final comparison, it compares our input with the calculated number in the first set of checks, so 90 == 90. Since they are equal, the function returns 0 for success.
Despite any char input for the check(int32_t * array, (int32_t) input), the result will be the same regardless because of the pre-set values in the array. What can I do to correct this?
