[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sun, 18 Dec 2011 22:56:34 +0100
From: magnum <john.magnum@...hmail.com>
To: john-dev@...ts.openwall.com
Subject: Re: 1.7.9-jumbo
On 12/18/2011 08:22 PM, Solar Designer wrote:
> On Sun, Dec 18, 2011 at 07:41:07PM +0100, magnum wrote:
>> Memory comes back to me now... for the non-utf8 versions of set_key(),
>> the 1.7.8 code was buggy. It truncated at 28 instead of 27 (but this was
>> only a problem when running external mode). I have a patch somewhere
>> that I can post soon, that re-introduces the tests as if external.c was
>> not truncating, but that does it in a more effective way - and properly
>> at max length. And if 1.7.8 would still be a tad faster after that, I
>> will just say that's because it was buggy.
>
> Are you saying that 1.7.8-jumbo's benchmark results for NT were
> inflated because of a bug (I don't see how this can be the case...),
> and that this has been corrected?
I'm sorry for the confusion. I did not recall the details so I thought
the code in J5 was completely without length checks. I have checked
current code, some git history and my sore brains and I believe this is
what happened:
1. I found out NT had a truncation bug that only showed up in external
mode (it truncated at 28).
2. I fixed the truncation bug in NT and asked how to do with external.c
as I thought that would be a smarter fix.
At this point I could not do much about the performance drop, without a
truncating external mode.
3. I implemented truncation in external.c (before your answer, since
dynamic and other formats really depend on it)
4. I dropped the new length check (so there was NO length check
anywhere) and expected a boost but to my surprise that was even slower!
So I left it with the correct (but no longer needed) truncation and
forgot all about it (at this point I should have reverted the non-utf8
set_key() changes before posting the patches).
5. Jumbo-5 was released with the code as in (2) above.
Now, we can choose to revert set_key() and set_key_encoding() to the
"buggy" versions in 1.7.8 - the bug is now harmless as long as external
truncates. But I would much rather figure out why the speed went from
26500K (original code) to 24000K (no length check at all) by completely
removing the extra condition in the for loop. [These figures are for my
intel laptop, the J5 code (correct length check) is 25500K]. How can a
dropped conditional make a significant performance drop?
>> For set_key_utf8(), the code present in jumbo-5 has got to be faster
>> than 1.7.8, and now properly truncates at 27. In UTF-8 mode we must
>> always take care of truncation since the "reported" max length is 3x27.
>> Try running "-test -enc:utf8" comparing 178J8 and 179J5, that's a quick
>> test only including Unicode formats.
>
> 1.7.8-jumbo-8:
> $ ./john -te -fo=nt -enc=utf8
> Benchmarking: NT MD4 [128/128 X2 SSE2-16] in UTF-8 mode... DONE
> Raw: 21421K c/s real, 21421K c/s virtual
>
> 1.7.9-jumbo-5:
> $ ./john -te -fo=nt -enc=utf8
> Benchmarking: NT MD4 [128/128 X2 SSE2-16] in UTF-8 mode... DONE
> Raw: 20427K c/s real, 20427K c/s virtual
>
> (I repeated both runs twice, as usual - got similar results.)
>
> So 1.7.8-jumbo-8 reports slightly faster speed even for utf8.
IIRC it went from like 21000K to 25000K on the AMD when I produced that
patch, I was pretty happy with it. My intel laptop show a smaller boost,
from 20500K to 21500K. I too repeated my tests. This is what I meant
with "compiler randomness". BTW I use gcc-4.6 on the laptop and it may
have been gcc-4.5 on the AMD.
Besides, the UTF-8 change is still needed, regardless of external.c
magnum
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
