Date: Thu, 24 May 2012 14:33:26 -0500 From: "jfoug" <jfoug@....net> To: <john-dev@...ts.openwall.com> Subject: RE: memory usage within JtR and possible ways to significantly reduce it. The hot/cold does makes a lot of sense. Very good way to try to keep the locality. For a 10 million candidate search, you still have 40mb of hot memory and 120mb of 'cold' memory, vs 160mb of arbitrary accessed memory. I am not sure that reducing from 160mb to 40mb will have that 'huge' of a help, but it might. But I certainly understand the goal, and it certainly should not hurt overall memory usage. Did you have any problems with starting on the source() (or rebuild_hash() or whatever), within the current jumbo john (prior to 1.8), just to start working through any unforeseen issues? If you have no problem, then what interface would you like to use, so that I could start on that, and have built using the interface you would like. I am not looking at starting to split the binary (just yet), but am looking at starting on the 'optional' source() method, to eliminate having to have the hashes allocated (IF a format can recreate the hash, some may not be able to do that)? Jim. >From: Solar Designer [mailto:solar@...nwall.com] > >BTW, a related idea (that's been on my to-do for a couple of years) is >to split the allocations into hot and cold. With the current formats >interface, binary would be hot and source would be cold. (This split >into hot/cold is not implemented yet, but it could be.) However, with >the introduction of source(), it seems that we also need to split binary >into hot and cold parts (would be 4 and 12 bytes in the example above). >The intent is to improve the locality of reference.
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.