Date: Sun, 18 Dec 2011 22:56:34 +0100 From: magnum <john.magnum@...hmail.com> To: john-dev@...ts.openwall.com Subject: Re: 1.7.9-jumbo On 12/18/2011 08:22 PM, Solar Designer wrote: > On Sun, Dec 18, 2011 at 07:41:07PM +0100, magnum wrote: >> Memory comes back to me now... for the non-utf8 versions of set_key(), >> the 1.7.8 code was buggy. It truncated at 28 instead of 27 (but this was >> only a problem when running external mode). I have a patch somewhere >> that I can post soon, that re-introduces the tests as if external.c was >> not truncating, but that does it in a more effective way - and properly >> at max length. And if 1.7.8 would still be a tad faster after that, I >> will just say that's because it was buggy. > > Are you saying that 1.7.8-jumbo's benchmark results for NT were > inflated because of a bug (I don't see how this can be the case...), > and that this has been corrected? I'm sorry for the confusion. I did not recall the details so I thought the code in J5 was completely without length checks. I have checked current code, some git history and my sore brains and I believe this is what happened: 1. I found out NT had a truncation bug that only showed up in external mode (it truncated at 28). 2. I fixed the truncation bug in NT and asked how to do with external.c as I thought that would be a smarter fix. At this point I could not do much about the performance drop, without a truncating external mode. 3. I implemented truncation in external.c (before your answer, since dynamic and other formats really depend on it) 4. I dropped the new length check (so there was NO length check anywhere) and expected a boost but to my surprise that was even slower! So I left it with the correct (but no longer needed) truncation and forgot all about it (at this point I should have reverted the non-utf8 set_key() changes before posting the patches). 5. Jumbo-5 was released with the code as in (2) above. Now, we can choose to revert set_key() and set_key_encoding() to the "buggy" versions in 1.7.8 - the bug is now harmless as long as external truncates. But I would much rather figure out why the speed went from 26500K (original code) to 24000K (no length check at all) by completely removing the extra condition in the for loop. [These figures are for my intel laptop, the J5 code (correct length check) is 25500K]. How can a dropped conditional make a significant performance drop? >> For set_key_utf8(), the code present in jumbo-5 has got to be faster >> than 1.7.8, and now properly truncates at 27. In UTF-8 mode we must >> always take care of truncation since the "reported" max length is 3x27. >> Try running "-test -enc:utf8" comparing 178J8 and 179J5, that's a quick >> test only including Unicode formats. > > 1.7.8-jumbo-8: > $ ./john -te -fo=nt -enc=utf8 > Benchmarking: NT MD4 [128/128 X2 SSE2-16] in UTF-8 mode... DONE > Raw: 21421K c/s real, 21421K c/s virtual > > 1.7.9-jumbo-5: > $ ./john -te -fo=nt -enc=utf8 > Benchmarking: NT MD4 [128/128 X2 SSE2-16] in UTF-8 mode... DONE > Raw: 20427K c/s real, 20427K c/s virtual > > (I repeated both runs twice, as usual - got similar results.) > > So 1.7.8-jumbo-8 reports slightly faster speed even for utf8. IIRC it went from like 21000K to 25000K on the AMD when I produced that patch, I was pretty happy with it. My intel laptop show a smaller boost, from 20500K to 21500K. I too repeated my tests. This is what I meant with "compiler randomness". BTW I use gcc-4.6 on the laptop and it may have been gcc-4.5 on the AMD. Besides, the UTF-8 change is still needed, regardless of external.c magnum
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.