john-users - Re: Patch: Improved raw-md5 and new format (Post.Office MD5)

Openwall wordlists collection for password cracking (20+ languages)
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 21 Nov 2005 16:11:58 +0300
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: Patch: Improved raw-md5 and new format (Post.Office MD5)

David,

I am really sorry it's taken me so long to get back to you on this.

On Wed, Oct 26, 2005 at 05:26:30PM +1000, David Luyer wrote:
> I'd like to submit the attached patch to John the Ripper.
> Please feel free to include the patch on the website or in
> any future version (it is released under the terms of GPL
> version 2).

I've just placed the patch into this FTP directory:

ftp://ftp.openwall.com/pub/projects/john/contrib/

under the name john-1.6.39-rawmd5-pomd5-1.diff.gz (notice that I've
added a "-1" for patch revision number).

I'd like to also link it off the webpage, but for that I prefer it to be
split back into two separate patches for the different hash types.

> The patch:
> 
> 1.	Improves the performance of raw-md5 around 20% compared
> 	to the version currently on the website. [1]

That's nice.  A lot of further improvements are possible indeed; maybe
I'll redo this stuff myself if I ever integrate it into John. :-)

> 2.	Implements the Post.Office MD5 crypt format, which may
> 	be found in Post.Office and in LDAP directories migrated
> 	from Post.Office (for example, OpenWave and qmail-ldap
> 	can support this crypt format).

Please excuse my ignorance, but what's Post.Office?  Unfortunately,
Google doesn't want to leave the dot included in the search string, so I
am getting a lot of results for various other post offices.

> [3] Earlier "after" test runs were at 3100K c/s to 3200K c/s
>     however this was due to an error in the memset() call
>     which didn't show up as the memory was usually already zero.
>     It may be possible to improve performance towards these
>     levels if the memory can be zero'd faster than it is by
>     memset(), for example perhaps a memcpy from memory which
>     is already zero?

No, memcpy() is unlikely to be faster.  There are other techniques which
might be faster on many systems (e.g., direct assignments to 64-bit
variables, especially interleaved integer/floating-point ones), but
slower on some others.

I do not think you should try to push this pure C non-parallelized
implementation to its limits.  Clearly, greater speedups are possible by
interleaving instructions from 2+ instances of MD5 (like it is done in
MD5_std.c) on non-register-starved archs (that is, not on x86) and by
doing 2+ instances of MD5 in parallel on vector registers if those are
available.  In fact, people have been doing the latter already.

Thanks,

-- 
Alexander Peslyak <solar at openwall.com>
GPG key ID: B35D3598  fp: 6429 0D7E F130 C13E C929  6447 73C3 A290 B35D 3598
http://www.openwall.com - bringing security into open computing environments