Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sun, 27 Feb 2011 17:53:18 -0600
From: "jfoug" <>
To: <>
Subject: RE: MD5 Generic improvements

-----Original Message-----
>I have been working offline with Solar recently, due to benchmark testing
>being broken for the md5-gen format.  The issue was found, and simple fix
>addressed it in the jumbo-12 update.
>I had a pretty large set of semi completed code improvements to md5-gen.  I
>have taken the time to finalize these, and to add quite a few additional
>things.  There are performance improvements (including intrinsic support,
>the intrinsic patch is applied), and some overall performance improvements,
>and certain sub formats improved significantly.

Version 2 of this 'early' patch, is now posted on the wiki patch page.

This patch, is against a vanilla john-1.7.6-jumbo-12 version.

Fixed in this patch:

1. Saltless hashes were appearing to be salted, and running SLOW. This
happened if the hash input in the file had more data after the password

2. Several of the primitive functions had bugs in them. Some were broken,
were not in use in any 'existing' format. Some were broken only for MMX or
and SSE.  Some were broken only if data was being appended to a non-empty
buffer, and the buffer had to have a certain number of bytes in it.

3. Integration of a 'thin' saltless format was not ideal.  Now, if the salt
length in the format is 0, the format will behave without a salt, no matter

Additional new functionality in this patch

1. Added code to switch into and out of SSE.  This will allow some work to
de done using the faster SSE, when it is safe, and then to switch in to 
MD5_go code, when it is no longer safe.  Safe for SSE is where all data 
fits into 1 MD5 'limb'.  That means 54 bytes of data is the most that can
be done.  So, md5(md5($s).md5($p)) will have some data that is known to be
64 bytes long (the md5 of the 2 32 byte strings concatenated). Thus, with
change, much of the MD5's can be done using SSE, and only the final MD5 call

has to be done using MD5_go.

2. Made changes so that if you specify, exactly which format to test, the 
md5-gen code will try to work with 'raw' hashes.  Thus, a file of 32 byte
base-16 numbers, can be processed for md5($p) by doing:
  john -format=md5-gen -subformat=md5_gen(0)
or with
  john -format=raw-md5
They will both do the same thing.
However, john -format=md5-gen -subform=md5_gen(2)  will test for
and john -format=md5-gen -subform=md5_gen(3)  will test for
on the exact same input file, without having to hand edit anything
This was a user request.  I thought it would be harder than it was. In the
end it was not hard at all, and gives the md5-gen format more user
NOTE, this will likely ONLY be useful for non-salted hashes within john.


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.