Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date: Tue, 22 Apr 2014 03:01:10 +0400
From: Solar Designer <>
Subject: Re: mmap(2) in wordlist.c

On Mon, Apr 21, 2014 at 11:56:03PM +0200, magnum wrote:
> On 2014-04-21 21:22, Solar Designer wrote:
> >I think this is worth a try, while also keeping the present approach
> >(for Windows, and for systems where it's faster).
> That clashes with a hidden agenda though: A total rewrite of wordlist.c 
> starting with core 1.8 version, and re-implementing Jumbo features as 
> cleanly and KISS as possible. Unless real-world performance in some 
> situations end up a lot worse than today's, I'd like to drop the old 
> buffering altogether, including the --mem-file-size option which ideally 
> should not be needed.

Right.  When I wrote "also keeping the present approach" I didn't mean
the present implementation of it in jumbo.  And the approach may change
somewhat, too.  I only meant that we need both a read() and a mmap()
version of the code.

> >I briefly experimented with read() vs. mmap() while developing popa3d,
> >in 1998 or 1999.  At the time, read() turned out to be significantly
> >faster, on Linux 2.0.x and ext2fs, when the read buffer size is properly
> >tuned.  I didn't try this experiment in JtR.
> What order of size was that at the time?

64 KB.  I think somewhat larger worked OK too, but wasn't necessary.
popa3d still has it at 64 KB, I didn't bother re-tuning (and this might
still be nearly optimal anyway).


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.