Date: Fri, 21 Aug 2015 00:31:38 +0300 From: Solar Designer <solar@...nwall.com> To: john-dev@...ts.openwall.com Subject: Re: BENCHMARK_LENGTH bugs On Thu, Aug 20, 2015 at 07:30:17PM +0200, magnum wrote: > On 2015-08-20 19:13, Kai Zhao wrote: > >I create a patch for benchmark_length: > > > >https://github.com/loverszhaokai/JohnTheRipper/commit/810a62c938d86a724f5398d312bbb1c5736a1147 > > >Testing: 7z, 7-Zip (512K iterations) [SHA256 AES 32/64]... (8xOMP) > >FAILED (This format is very slow, but it will report separate "Many > >salts" vs. "Only one salt" speeds) > > Your patch has a hard-coded list of "slow hashes". That's not good from > any point of view. I agree that this kind of a hard-coded list isn't helpful. If these are the only formats for which warnings may be printed, then we could as well just review them manually and correct whatever wrong settings there might be. > As seen in the above figures, you should do calculate > a figure many/one like this: > > >$ ../john --test --format=7z > > > >Will run 8 OpenMP threads > >Benchmarking: 7z, 7-Zip (512K iterations) [SHA256 AES 32/64]... (8xOMP) > >DONE > >Speed for cost 1 (iteration count) of 524288 > >Many salts: 3056 c/s real, 3034 c/s virtual > >Only one salt: 18.0 c/s real, 18.1 c/s virtual > > 3056/18 = 169.78x faster. Definitely nothing to complain about. > > >$ ../john --test --format=dominosec8 > > > >Will run 8 OpenMP threads > >Benchmarking: dominosec8, Lotus Notes/Domino 8 [8/64]... (8xOMP) DONE > >Warning: "Many salts" test limited: 1/256 > >Many salts: 3471 c/s real, 435 c/s virtual > >Only one salt: 3303 c/s real, 416 c/s virtual > > 3471/3303 = 1.05x so a 5% speedup. I'm not sure we should complain even > here. > > I think you should complain if speedup is less than 1-2%. Also, I'm not > sure you should FAIL but maybe just "Warning: ..." like with alignment > issues. Solar? The threshold should be way below 1%. We do want to see separate results if they are expected to differ by 1% or more. Unfortunately, this prevents reliable automatic detection. It is possible that on a given run results just happen to differ by 2% even if there's no good reason for them to differ (a slight change in system load may cause this), or on the contrary are almost the same (difference below 1%) even if they are expected to differ by a few percent. Yet we could try detecting and warning about Many / One salt results that are too similar (difference less than 1%). Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.