[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Thu, 24 May 2012 14:33:26 -0500
From: "jfoug" <jfoug@....net>
To: <john-dev@...ts.openwall.com>
Subject: RE: memory usage within JtR and possible ways to significantly reduce it.
The hot/cold does makes a lot of sense. Very good way to try to keep the
locality. For a 10 million candidate search, you still have 40mb of hot
memory and 120mb of 'cold' memory, vs 160mb of arbitrary accessed memory. I
am not sure that reducing from 160mb to 40mb will have that 'huge' of a
help, but it might.
But I certainly understand the goal, and it certainly should not hurt
overall memory usage.
Did you have any problems with starting on the source() (or rebuild_hash()
or whatever), within the current jumbo john (prior to 1.8), just to start
working through any unforeseen issues? If you have no problem, then what
interface would you like to use, so that I could start on that, and have
built using the interface you would like.
I am not looking at starting to split the binary (just yet), but am looking
at starting on the 'optional' source() method, to eliminate having to have
the hashes allocated (IF a format can recreate the hash, some may not be
able to do that)?
Jim.
>From: Solar Designer [mailto:solar@...nwall.com]
>
>BTW, a related idea (that's been on my to-do for a couple of years) is
>to split the allocations into hot and cold. With the current formats
>interface, binary would be hot and source would be cold. (This split
>into hot/cold is not implemented yet, but it could be.) However, with
>the introduction of source(), it seems that we also need to split binary
>into hot and cold parts (would be 4 and 12 bytes in the example above).
>The intent is to improve the locality of reference.
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
