It just a normal 32 bit application unpacked. So we just need to fire up our debugger to analyse it.
I used 32 bit version of x64dbg to reverse engineer this program. Run the program in the debugger and just dummy input to get the output.
So we get "Wrong" output. We check those output string in strings references.
Here we got 2 interesting strings which "Correct!!" and "Wrong!". So I just right click in "Correct!!" string and follow in dissambler to go to the address that display it.
Then check how this program worked. Before it go to the output whether it is correct or wrong, we can see that there are some algorithm checking the input.
After going through the algo, it's just a simple algo which done with xor, shift right and rotate right. Lastly it will compare the input with 0xF298DC9E. So we know that our input must be exact the same with 0xF298DC9E to get the correct output. You can see that at address 0x00251112.
So i just make my own program in C that act same like the program.
Run the program with the value a = 0. Variable 'a' here same with EAX register in the program. And also we run the real program also in our debugger with the input 0. But before that I had mark breakpoint at the comparing section in address 0x00251112.
You can see that my program output same with the debugged program! The output is 0xE098D6DA. We know the algo, so we just need to reverse it now.
The problem when I want to reverse it is when I reached at
a = (c ^ a) + c + a;
Because I dont know where to get the value for variable 'c'. My idea to solve this problem is just by bruteforce it!(If you know the better way to reverse it, just comment below.) Haha.
Here are my code in c.
Okay in this code, i just do looping for ULONG_MAX which is equals to 4294967295 or 2^32 - 1. Why using 32bit? This is because we want to bruteforce the value in EAX register, as we know EAX register hold 32 bits data which equals to 4 bytes.
After going through all the code, lastly it will compare variable 'a' with 0xF298DC9E, if it is true then it will display our answer!
So we got our answer which are 77859328. We try it with the real program.
Got "Correct!!". Yeahh. So that's our flag.
It was really fun to solve this challenge since I am too newbies in this area, and I know my explanation not so good like other write up since my understanding in assembly language, reverse engineering and other technical part are so lowwwwww...
References :
1) http://feetsonmyshoes.blogspot.my/2012/04/uniten-hack10-2012-write-up-ex02.html
2) http://justanotherctfnewbie.blogspot.my/2015/11/utphax15-group-stage-round-1-binary.html