Date: Tue, 15 Sep 2015 18:22:41 +0300 From: Solar Designer <solar@...nwall.com> To: john-dev@...ts.openwall.com Subject: Re: Judy array On Tue, Sep 15, 2015 at 09:45:51AM +0200, magnum wrote: > On 2015-09-15 03:03, Solar Designer wrote: > >magnum - testing this stuff, I see that pot sync is a major bottleneck. > > Does it look like pot sync is slower than processing "our own" cracks? Yes, it's a lot slower than processing of the process' own cracks. However, I didn't compare it against processing of john.pot records on loading. > It shouldn't, it uses all the same gears in the box. Of course, with > -fork=8 it will spend 8x more time processing cracks, It appeared way worse than 8x in my testing. With a large pot file buffer, the processes would become unresponsive for tens of seconds and would move way down in "top", leaving idle CPU time (weird). Oh, maybe that's because you acquire a lock on the pot file while reading it? You shouldn't need to: you may instead lock, save the file size in a variable, unlock, and then read up to the recorded size (it'd be a whole number of pot file records then). Or you may read without locking at all, as long as you ignore the last partial line (easy to detect: if it's not containing a colon, it's not complete enough for pot sync). > but in real-life it's not a problem. This is a contrived test. A contrived test, yes, but if we manage to handle even contrived cases like this well, we might end up improving handling of more common cases as well. > >Since this is your feature, you might want to benchmark and optimize it > >some more, or/and maybe we just want it disabled by default when cracking > >saltless hashes. As it is, it's just not able to handle Fred's "about > >1.7 million solutions" in a minute - it takes several minutes to process > >those guesses, so it becomes the primary bottleneck for the whole run. > > I'll verify that we don't have some regression in there, I did spend a > lot of time optimizing it. Thanks! Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.