[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Sun, 27 Apr 2014 9:44:33 -0400
From: <jfoug@....net>
To: john-dev@...ts.openwall.com
Subject: Re: mmap()
>From an offlist request. I thought this might be good information for the list, to see how cygwin, which is still limited to the diskIO constraints of Win32 (which SUCK) behaves. It does work with magnums mmap port (had to adjust some defines in the include section, but it was no big deal). I have no idea how real-world ready this code is at the moment. magnum has this in a separate tree in git, so I assume it is still being actively developed, and not ready for real world.
My testing:
Cygwin testing (on crappy laptop). pw file is 1 rawmd5 at very end of rock-u passwords
Command line:
$ ../run/john in -form=raw-md5 -w=/dictionaries/RockYou_UserAccount-passwords.txt
$ wc /dictionaries/RockYou_UserAccount-passwords.txt
7457743 7503103 70833938 /dictionaries/RockYou_UserAccount-passwords.txt
Testing with mmap:
Loaded 1 password hash (Raw-MD5 [MD5 128/128 SSE2 12x])
jucarius (u0-RawMD5)
1g 0:00:00:02 DONE (2014-04-27 08:09) 0.4452g/s 3320Kp/s 3320Kc/s 3320KC/s 1616888..jucarius
Testing with bleeding (no memory buf)
1g 0:00:00:04 DONE (2014-04-27 08:14) 0.2157g/s 1609Kp/s 1609Kc/s 1609KC/s 1616888..jucarius
Testing with bleeding (memory buf large enough to hold entire file)
1g 0:00:00:01 DONE (2014-04-27 08:17) 0.5341g/s 3983Kp/s 3983Kc/s 3983KC/s 1616888..jucarius
With rules: (Note, input hash busted, so the full rules would run)
Testing with mmap:
0g 0:00:01:04 DONE (2014-04-27 08:21) 0g/s 1950Kp/s 1950Kc/s 1950KC/s Ddiinnkkoing..Jucariusing
Testing with bleeding (no memory buf)
0g 0:00:03:56 DONE (2014-04-27 08:27) 0g/s 535200p/s 535200c/s 535200C/s Ddiinnkkoing..Jucariusing
Testing with bleeding (memory buf large enough to hold entire file)
0g 0:00:00:51 DONE (2014-04-27 08:22) 0g/s 2479Kp/s 2479Kc/s 2479KC/s Ddiinnkkoing..Jucariusing
For comparison:
$ ../run/john -test=5 -form=raw-md5
Benchmarking: Raw-MD5 [MD5 128/128 SSE2 12x]... DONE
Raw: 13875K c/s real, 13876K c/s virtual
So mmap was 83% of mem buffer for a flat run and 78.6% speed for rules. But even the fastest run was only 28.7% the speed of bench, so we are currently sucking a lot of hot air with wordlist mode.
Jim.
---- magnum <john.magnum@...hmail.com> wrote:
> Could you try checking out the "mmap-array" branch and see if it builds
> at all under Cygwin and/or VC or MinGW. It uses mmap() and I believe
> Cygwin will handle it as-is, while non-Cygwin will need a wrapper that
> is included but not tested. The wrapper is .c but is #included in
> wordlist.c (when "windows but not cygwin") so inlined.
>
> So first question is, does it build at all? Second question is, how's
> performance?
>
> I have tested following on OSX and Linux:
>
> 1. Rockyou list against a single NT hash
> 2. Rockyou list against a single NT hash + rules
> 3. Rockyou list against a single NT hash + rules and --mem=0
> ...plus all above cases with --fork=4
>
> In all cases this code is faster, or as fast. In some cases it's 50-200%
> faster than current bleeding.
>
> There is also SSE2 code alternative for the mgetl() [which is a mmap
> version of fgetl()] but it's commented out for now. It boosted Linux a
> little but totally ruined performance on OSX. Actually that was probably
> not because of OS but because of different CPUs. Some CPUs have a lot of
> penalty for unaligned SSE2, others have more or less none. Anyway the
> boost on Linux was so small I'll probably just scrap that part.
>
> magnum
>
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
