Date: Wed, 8 Jun 2011 16:54:09 -0500 From: "jfoug" <jfoug@....net> To: <john-dev@...ts.openwall.com> Subject: RE: unique I have not had a chance to look at unique. I know it will be slower, since it has to deref memory several times, millions of times per second. That's not free. But the slow down was pretty trivial. However, the speed up is an order of magnitude, to several orders of magnitude improvement, IF someone knows how and when to tweak the memory. For little wordlists (couple mb to a few dozen mb), the difference was insignificant. For 100mb, there was a tiny notice in 'default' runtime, but not too bad. Also, that may be the point where someone would want to kick the mem up a little. At 500mb, the slowdown for the 'default' should not matter, since john should be upp'ing the size anyway. What I would love to add, but which is a little more fraught with error, is to detect file size, detect memory size (and try to detect if there are quotas), and auto advance mem usage when it 'should' be done. If we take this case, it would have to be a little conservative, but would be more than 'default'. But before going off 1/2 cocked on a change like this, it should be chatted about a bit, because this may have additional unwanted side effects. Jim. >-----Original Message----- >From: Solar Designer [mailto:solar@...nwall.com] >Sent: Wednesday, June 08, 2011 4:15 PM >To: john-dev@...ts.openwall.com >Subject: Re: [john-dev] unique > >Jim - > >On Sat, Jun 04, 2011 at 08:42:31PM +0400, Solar Designer wrote: >> As to optimizing this, you can easily pre-compute "vUNIQUE_HASH_SIZE - >1" >> and "vUNIQUE_HASH_LOG / 2". >... >> Here's a marketing workaround: double the default memory usage by >unique >> when the jumbo patch is applied (change UNIQUE_HASH_LOG from 20 to 21, >> UNIQUE_BUFFER_SIZE from 0x4000000 to 0x8000000). > >I am making the above changes for -jumbo-6. > >Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.