Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Mon, 3 Sep 2012 23:12:18 -0400 (EDT)
Subject: RE: memory usage (was: [john-users] JTR against 135
 millions MD5 hashes)

Thank you for the long and insightful write up.

On Mon, Sep 3, 2012 at 8:49 PM, Solar Designer wrote:
> Jim -
> Thank you for sharing your thoughts on this.
> On Mon, Sep 03, 2012 at 11:16:44AM -0400, wrote:
>> The memory reduction due to the source() function, and also due to 
>> removing some wasteful allocations within the prepare() and valid() 
>> within dynamic.  I believe the wasteful allocations in dynamic have 
>> been removed, all the way back to the J7-RC.   The memory 
>> requirements were greatly reduced early on, but have slowly started 
>> to be lost some (the bitmaps).  However, total mem usage is still 
>> much less than it was, when we were keeping the full source buffer 
>> for each hash.
> I did not realize that there were wasteful allocations in prepare() 
> and
> valid().  Weren't they temporary?

They were done using alloc_memory_tiny.  Now, I simply use static 
buffers.  When raw hashes were being used, and a few other cases, there 
could be one or more of these type allocations, which are very similar 
to memory leaks, when done on temp transient data.

> Another potential source of memory usage reduction are the alignments.
> For raw MD5 hashes, a 4-byte alignment should suffice (they're 4x4
> bytes), yet we were using 8-byte alignment on 64-bit builds.

Very good point. I had not even thought of things like these new 
alignment requirements.  I really think for most formats, 4 is the 
correct alignment, due to dereference of an arch_word_32 in some of the 
lookup functions.

I do not have time to fully address all of the issues brought up in your 
reply, but will try to look more in depth at this tomorrow afternoon.


Powered by blists - more mailing lists

Your e-mail address:

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.