john-dev - RE: Question over validity of concept in JtR's benchmarking code.

Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Thu, 8 Sep 2011 16:27:58 -0500
From: "jfoug" <jfoug@....net>
To: <john-dev@...ts.openwall.com>
Subject: RE: Question over validity of concept in JtR's benchmarking code.

Here is the quick'n'dirty hack I made to bench to get non-correct passwords
(for my testing).  I will both attach, and put this code inline.

 

diff -urpN john-1.7.8-jumbo-6a/src/bench.c john-1.7.8-jumbo-6b/src/bench.c

--- john-1.7.8-jumbo-6a/src/bench.c      2011-08-03 16:19:13.000000000 +0000

+++ john-1.7.8-jumbo-6b/src/bench.c   2011-09-08 19:35:08.828125000 +0000

@@ -82,9 +82,12 @@ static void bench_handle_timer(int signu

static void bench_set_keys(struct fmt_main *format,

               struct fmt_tests *current, int cond)

{

-              char *plaintext;

+             char *_plaintext;

+             static char *plaintext;

               int index, length;

+             if (!plaintext) plaintext = mem_alloc_tiny(4096,
MEM_ALIGN_WORD);

+

               format->methods.clear_keys();

                length = format->params.benchmark_length;

@@ -92,7 +95,9 @@ static void bench_set_keys(struct fmt_ma

                               do {

                                               if (!current->ciphertext)

                                                               current =
format->params.tests;

-                                              plaintext =
current->plaintext;

+                                             _plaintext =
current->plaintext;

+                                             strcpy(plaintext, _plaintext);

+                                             *plaintext ^= 0x22;

                                               current++;

                                                if (cond > 0) {

 

 

Here were some timings:

 

Real world, single salt   4450k/ to 4580k/s  Multiple salts (9), 6980k/s to
7280k/s

 

The testing was done using inc:alnum.  The most notable difference in speed,
was 1 byte checksum vs 2 byte checksum (the 2 byte checksum removes more bad
passwords, quicker). 

 

The 'original' benchmark testings are:

 

john -test=5 -for=pkzip

Benchmarking: pkzip [N/A]... DONE

Many salts:     47204 c/s

Only one salt:  47616 c/s

 

 

I can get this 'closer' to reality. If I removed stages 3 and 4 (the calls
into zlib), I ended up with these timings:

john -test=5 -for=pkzip

Benchmarking: pkzip [N/A]... DONE

Many salts:     3773K c/s

Only one salt:  3021K c/s

 

However, if I keep the format exactly the same way it currently is (all 4
'levels' inside of crypt), but change bench.c to not give the correct
password, here is the timings:

john -test=5 -for=pkzip

Benchmarking: pkzip [N/A]... DONE

Many salts:     7437K c/s

Only one salt:  4805K c/s

 

These are the CORRECT timings, which very closer represents real running.
The reason the format is slower, is the overhead in creation of the
passwords, AND due to some of them passing the checksum.

 

Jim.

 

From: jfoug 

 

Is sending correct passwords to the benchmark functions a 'valid' way to
benchmark?    In real testing, we assume that a correct password, is a VERY
rare event. There may be billions, or 100's of billions of passwords tested,
before a correct password is found (if ever).  

 

Would it not make more sense, to provide incorrect passwords when
benchmarking the format?

 


Content of type "text/html" skipped

Download attachment "bench-fixes.diff" of type "application/octet-stream" (914 bytes)
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.