Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sun, 19 May 2013 03:21:42 +0400
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: Incremental mode in 1.7.9.14

On Sun, May 19, 2013 at 02:57:00AM +0400, Solar Designer wrote:
> I've attached the better version of the charset.c patch.  Please test.

Here's what I am getting with it, vs. 1.7.9's incremental mode with the
same CHARSET_* settings and the same training and test sets.

For my testing, I ran "shuf" on the full RockYou list (32.6M passwords).
Then I split the resulting shuffled list in two: exactly 1M for test
set, and the rest (about 31.6M) for training set.  There's no overlap
between the two, but some of the passwords that occurred in the original
RockYou list more than once also occur in both sets.  This simulates
what happens with common passwords across two different yet similar
sites, where cracked (or otherwise known) passwords from one are then
used to attack hashes from the other.

$ shuf r > rs1
$ head -1000000 rs1 > rs1a
$ tail +1000001 rs1 > rs1b
$ wc -l rs1?
  1000000 rs1a
 31603388 rs1b
 32603388 total
$ sed 's/^/:/' < rs1b > john.pot
$ perl -ne 'chomp; print "dummy:", "\$dummy\$", unpack("H*", $_), "\n";' < rs1a > pw1

#define CHARSET_MIN			0x01
#define CHARSET_MAX			0xff
#define CHARSET_LENGTH			24

JtR 1.7.9 took almost 4 minutes to generate a .chr file with these
settings from the .pot file above.  The new version with the patch I
posted takes under 50 seconds to do the same (indeed, the .chr file is
different - it's for the new version, too).  This is on Xeon E5420
(using one core).

JtR 1.7.9 cracking run with status printed after 1k, 10k, 100k, 1M, 10M,
100M, 1G candidates tested:

guesses: 69  time: 0:00:00:01  c/s: 80604K  trying: 123456 - sanie13
guesses: 2072  time: 0:00:00:05  c/s: 1377M  trying: momesta1 - 07706101989
guesses: 8402  time: 0:00:00:10  c/s: 6835M  trying: bisnon - march
guesses: 27151  time: 0:00:00:16  c/s: 41802M  trying: buddelat1 - budante11
guesses: 70436  time: 0:00:00:29  c/s: 218147M  trying: mangstienter - mangstearina
guesses: 127439  time: 0:00:01:04  c/s: 915699M  trying: 281gg5 - 281gs4
guesses: 207516  time: 0:00:03:39  c/s: 2355G  trying: sk09927j - sk09383k

The new version with the patch I posted (for same candidate counts):

117g 0:00:00:00 140.9g/s 204.8p/s 204.8c/s 140815KC/s 123456..102526
2800g 0:00:00:02 1111g/s 3980p/s 3980c/s 3439MC/s sammys..shiess
11628g 0:00:00:05 2165g/s 18646p/s 18646c/s 13619MC/s amerior..amandom
41413g 0:00:00:10 3925g/s 94797p/s 94797c/s 66114MC/s loures..loutty
78550g 0:00:00:19 4024g/s 512299p/s 512299c/s 329525MC/s bbbba04..bbb2567
134074g 0:00:00:38 3467g/s 2585Kp/s 2585Kc/s 1523GC/s 163.180..163.c15
216565g 0:00:01:56 1861g/s 8595Kp/s 8595Kc/s 4388GC/s sedem92..sedee29

Another test, trained on 1000 passwords only:

$ head -1000 rs1b | sed 's/^/:/' > john.pot

yet cracking the same 1M dummy hashes as above.

JtR 1.7.9:

guesses: 30  time: 0:00:00:01  c/s: 91310K  trying: 123456 - *
guesses: 488  time: 0:00:00:05  c/s: 1378M  trying: 122212 - andrk6
guesses: 2282  time: 0:00:00:08  c/s: 8586M  trying: mosbit - motins
guesses: 8269  time: 0:00:00:11  c/s: 62020M  trying: cookrso11 - coolekean
guesses: 19663  time: 0:00:00:17  c/s: 396417M  trying: 9gce - 9g5k
guesses: 42935  time: 0:00:00:39  c/s: 1677G  trying: soerl36 - soel18o
guesses: 106099  time: 0:00:03:01  c/s: 3363G  trying: l0lbgnda - l0lbgoc0

New code:

27g 0:00:00:00 168.7g/s 1062p/s 1062c/s 730482KC/s acasandreigabriel..ashcer
931g 0:00:00:00 3210g/s 34586p/s 34586c/s 6890MC/s 10veey..180812
3910g 0:00:00:01 2917g/s 74723p/s 74723c/s 68589MC/s jeam90..jech05
10432g 0:00:00:02 4240g/s 406548p/s 406548c/s 340058MC/s ardr21..ard490
24578g 0:00:00:04 5026g/s 2045Kp/s 2045Kc/s 1674GC/s rpsamaz..rpsasol
53671g 0:00:00:13 3872g/s 7215Kp/s 7215Kc/s 4977GC/s p1rzi2a..p1rzers
114151g 0:00:01:17 1470g/s 12879Kp/s 12879Kc/s 7788GC/s thramne1..thram007

To get these status lines printed at the right times, I use a revision
of the AutoStatus external mode with these two lines added at the end of
its filter():

	abort = (interval == 1000000000);
	interval *= 10;

Alexander

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.