Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 31 Aug 2009 14:56:14 -0500
From: "JimF" <>
To: <>
Subject: Re: Thoughts and questions on creation of a 'generic' MD5 hash set format (to handle 'all' of them)

I have put patch file on the wiki at: which is the generic md5 code that I 
have done today.  The code should work for SSE2 or non-sse2 builds (sse2 is 
simply faster).

The format of a input file line is:


Here are a list of 'built-in' md5_gen(#) signatures, and exactly what 
expression they handle:

md5_gen(0) --> md5($p)   (raw md5).
md5_gen(1) --> md5($p.$s)
md5_gen(2) --> md5(md5($p))
md5_gen(3) --> md5(md5(md5($p)))
md5_gen(4) --> md5($s.$p)
md5_gen(5) --> md5($s.$p.$s)
md5_gen(6) --> md5(md5($p).$s)
md5_gen(7) --> md5(md5($p).$s) (vBulletin)  (fixed 3 byte salt, colon not 
valid as field sep since it can be in salt).
md5_gen(8) --> md5(md5($s).$p)
md5_gen(9) --> md5($s.md5($p))
md5_gen(10) --> md5($s.md5($s.$p))
md5_gen(11) --> md5($s.md5($p.$s))
md5_gen(12) --> md5(md5($s).md5($p)) (mybb)  // note will NOT work in SSE2 
md5_gen(13) --> md5(md5($p).md5($s))  // note will NOT work in SSE2 code
md5_gen(14) --> md5($s.md5($p).$s)
md5_gen(15) --> md5($u.md5($p).$s)      // note $u is not handled yet (but 
format is 'reserved')
md5_gen(16) --> md5(md5(md5($p).$s).$s2) // note 2 salts is not handled yet.
md5_gen(17) --> RESERVED
md5_gen(99) --> RESERVED
md5_gen(100) --> User specified (NOTE no way for user to do this 'yet', but 
I have code laid out to do this later)
md5_gen(101) --> User specified

Right now, only md5_gen(0) to md5_gen(99) are reserved.  We may want to bump 
that number up, but I think 100 is a pretty good number of hashes to 

Note, md5_gen(7) and md5_gen(6) are the same expression, but md5_gen(7) sets 
a specific 3 byte salt, and tells john that it MUST use a field separator 
other than ':'  (NOTE code as released does not force this, since there is 
no code in john yet, to allow anything other than a : to be a field 

Note, md5_gen(0) is standard 'raw-md5' and is a good test case for speed, vs 
the existing raw-md5 module.

Note, md5_gen(15) and md5_gen(16) are not implemented.  md5_gen(12) and 
md5_gen(13) will not function in an sse2 build (at this time), due to the 
SSE2 code only works correctly up to 54 bytes, and to get it working for 
these types would require 2 crypts, and also would slow down other sse2 
processing.  Thus, they simply do not work (but error out with a message 
listing that they are not supported on the SSE2 builds).

There is also a .zip file that contains many test files, and a password dic 
file to test them with. That makes easy work to validate that the formats 
ARE working once built.

NOTE john -test -format=md5-gen   does NOT work at this time (crashes), and 
john -test will also crash once it gets to md5-gen format.  I do know about 
this, I just have not dealt with how to 'work around' it at this time. 
This is due to the md5-gen format not being 'fully' initiallized, until it 
hits the first line from the input file (to obtain the signature to use). 
Once that signature is 'hit', then ONLY lines of that signature are deemed 
as 'valid'.  Thus, for testing, the 2nd init is happens, thus crash.  We 
might have to add a command line option to do something like

john -test -format=md5-gen -subformat=md5_gen(0)      (or something like 


To unsubscribe, e-mail and reply
to the automated confirmation request that will be sent to you.

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.