Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Thu, 02 Apr 2015 03:16:29 +0300
From: Alexander Cherepanov <ch3root@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: Advice on proposal: John the Ripper jumbo robustness

On 2015-04-01 05:13, Kai Zhao wrote:
>> The first thing to do with fuzzing john is making john start fast.
>
> What's the meaning of start fast?

Well, if you are interested in fuzzing specific format then you want to 
skip everything that doesn't affect your format while still reaching 
functions in your format which you are interested in.

If your format have a non-trivial valid() function (and I hope it holds 
for most valid()s in john by now) it will reject most samples that a 
fuzzer throws at it. Hence you can look at the time required to reject a 
bogus hash, e.g.:

$ echo garbage > test.pw
$ time ../run/john --format=7z test.pw
No password hashes loaded (see FAQ)

real	0m0.096s
user	0m0.084s
sys	0m0.008s

> Here is the main() function in john.c:
>
> int main()
> {
>      john_init();
>
>      john_run();
>
>      john_done();
> }
>
> Does "making john start fast" means "making john_init() fast"?

I've not looked into it in such details. Please research it.

>> I've taken a look at it. There are two easy things here: instrumenting
>> only a
>> fuzzed format and john.c make john ~5x faster, removing most content from
>> john.conf makes john ~7x faster. That's 35x boost.
>
> How do you get the multiple of ~5x and ~7x faster?

5x:

Fully instrumented build:

$ ./configure CC=.../path/to/afl-gcc && make
$ time ../run/john --format=7z test.pw
No password hashes loaded (see FAQ)

real	0m0.486s
user	0m0.476s
sys	0m0.008s

Instrumenting only necessary minimum:

$ ./configure && make && rm 7z_fmt_plug.o john.o && make 
CC=.../path/to/afl-gcc
$ echo garbage > test.pw
$ time ../run/john --format=7z test.pw
No password hashes loaded (see FAQ)

real	0m0.096s
user	0m0.088s
sys	0m0.004s

You have to instrument some format if you are interested in fuzzing it. 
And you have to instrument main() to get afl's fork server.


7x:

$ echo '[Options]' > john.conf
$ echo garbage > test.pw
$ time ../run/john --format=7z --config=./john.conf test.pw
No password hashes loaded (see FAQ)

real	0m0.015s
user	0m0.008s
sys	0m0.004s


The speed in afl jumped from <2 to 70-80 execs per second. It would be 
nice get further 10-20x speed-up.

To make sure that nothing is broken in the process you can compare 
samples that afl finds for a new build with the ones you found earlier.

-- 
Alexander Cherepanov

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.