Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Wed, 16 Sep 2015 02:05:30 +0300
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: Judy array

On Wed, Sep 16, 2015 at 12:43:44AM +0200, magnum wrote:
> On 2015-09-15 09:57, magnum wrote:
> >On 2015-09-15 05:40, Solar Designer wrote:
> >>This saves 4 seconds, but only when -mem=999999999 is also used:
> >>
> >>-WordlistMemoryMap = Y
> >>+WordlistMemoryMap = N
> >
> >We could probably default to not mmap when that other buffer is used.
> 
> I had a look, that's a no-go. When mmap was introduced the code was 
> simplified a bit. I'm not going to make it worse again. We still need to 
> re-write it from scratch (with "scratch" meaning the original non-Jumbo 
> file).
> 
> Also, I don't observe any gain from disabling mmap and only minimal gain 
> from using --mem=0 when mmap is enabled (I stopped using -mem after mmap 
> was implemented). I'm surprised by your results, especially given you 
> were short on memory... or is it *because* of just that..?

No, I wasn't that short on memory.  The machine has 16 GB, and after my
copy-on-write reducing fixes these test runs allocate at most 9 GB total
across the 8 processes.  So there's still enough memory to have the
wordlist, etc. cached in RAM across the runs, etc.

Let me re-test...

First, I took the latest cracker.c.  This got my prefetching disabled.
And, surprise, it's now 2 seconds slower (I still have SHR=0):

real    0m58.804s
user    4m26.433s
sys     0m18.934s

Maybe the prefetching is sometimes helping, after all, even though for
our current raw-md5 it's very limited (max_keys_per_crypt is only 12, so
those loops don't actually go to the hard-coded maximum of 64
outstanding prefetches).

Now, enabling mmap (still no prefetch, etc.):

real    1m0.847s
user    4m51.136s
sys     0m15.579s

In both cases, I also had -mem=0 on the command line.

Alexander

Powered by blists - more mailing lists

Your e-mail address:

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.