Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sun, 3 Jan 2010 23:36:27 +0300
From: Solar Designer <>
Subject: Re: JtR 1.7.4 and jumbo patch update

Regarding Matt's benchmark (of's rules engine vs. 1.7.4's),
which revealed a bug in 1.7.4:

On Sun, Dec 27, 2009 at 08:01:42PM -0500, Charles Weir wrote:
> A copy of the config file can be obtained from the following link:
> For the input dictionary I used one of the lowercase English
> dictionaries available on the openwall ftp site, (I think it was the
> large one). The dictionary contained 444,678 words.

/pub/wordlists/languages/English/4-extra/lower.gz contains 444,678 lines
(a few of which are comments rather than words), so I think it was this
one.  The -extra wordlists don't encompass the smaller and higher
quality ones, so they contain relatively obscure and questionable
"words" only.  It is not a good idea to use one of them on its own
(other than after having run through one or more of the higher quality
wordlists, such as those found under -large).  I thought this was
obvious from the naming ("extra" is just that), the comments in the
files, and the actual content, but perhaps this needs to be documented

Anyhow, the -extra wordlist is OK for a test run when there's no goal to
actually crack passwords.

> Running JtR version
> Ryoki:run cweir$ ./john
> -wordlist=../../../custom/dictionaries/english-lower -rules -stdout > /dev/null
> words: 10495949352  time: 0:01:04:26 100%  w/s: 2714K  current: 9zzzzzzzthi$
> ------------------------------------------------------------------
> Running JtR version 1.7.4
> Ryoki:run cweir$ ./john
> -wordlist=../../../custom/dictionaries/english-lower -rules -stdout > /dev/null
> words: 10495945056  time: 0:00:49:48 100%  w/s: 3512K  current: 9zzzzzzzthi$

> JtR 1.7.4 ran noticeably faster than JtR, completing its
> session in 76% of the time it took to finish. The one anomaly
> was that the 1.7.4 session outputted that it made 10,495,945,056
> guesses, while the session outputted that it made
> 10,495,949,352 guesses. The difference in guesses may have just been a
> reporting issue, (aka the final count might not be updated), but I'll
> leave it to someone more knowledgeable to answer that question.

As I pointed out before, this indicated that there was a bug somewhere,
and I provided a relevant patch, john-1.7.4-last-fix.diff, attached to
one of my postings.  Since there were no followups, I did not know
whether the above discrepancy was caused by the bug fixed by the patch
or not.  Today, I went to reproduce the issue myself, and I did.  For
this, I needed the -extra wordlist mentioned above (first 10k lines of
it proved to be enough), Matt's ruleset (with all instances of [0-9]
changed to [0-1] for quicker test runs), and a 64-bit build of JtR (the
problem was not reproducible with a 32-bit build).  The conclusion was
that john-1.7.4-last-fix.diff did not fix the bug (it fixed another
bug), and I came up with john-1.7.4-last-fix-bis.diff (attached to this
message) that fixes the problem identified in Matt's test runs above (at
least the way I reproduced it).

I've just released 1.7.4-jumbo-5 adding the extra hunk from
-last-fix-bis mentioned above, and indeed I am going to include a fix in
the next version of JtR.

BTW, I am not relying on the "word count" alone in my testing.  Rather
than redirect the output of JtR to /dev/null, I pipe it into md5sum, and
I make sure there are no unexpected changes in the MD5 digest of JtR's
output between versions.



View attachment "john-1.7.4-last-fix-bis.diff" of type "text/plain" (3172 bytes)

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.