Date: Fri, 25 Oct 2013 00:58:50 +0200 From: magnum <john.magnum@...hmail.com> To: john-dev@...ts.openwall.com Subject: Re: benchmark-unify and relbench adjustments On 2013-10-24 14:01, Frank Dittrich wrote: > magnum, Solar, all, > > changed format names and changed --test output require some adjustments > to increase the number of benchmarks which can be compared (between > bleeding and earlier versions, e.g., unstable. > Finally, I don't know yet how to handle this change: > > In addition to the format name, the format label has been included into > the output, at least if format label and format name differ. Format label is no longer all-lowercase and is always printed. Format name can now be empty but is printed if it isn't. > I was experimenting with another solution (this patch just checks an > environment variable instead of a new benchmark-unify option, but I can > provide a patch which introduces a --drop-labels option): > This will increase the number of benchmarks in bleeding that can be > compared with benchmarks in unstable, but it has unwanted side effects: > > -the user would need to run benchmark-unify even on the --test output of > the latest john version (so far I always aimed at converting older > output to the newest, and keeping the newest output unchanged) > > -in some cases, the format name will become (completely) meaningless > and/or ambiguous: Wouldn't the opposite be safer and easier: Only care about format label (anything up to first comma or bracket, and case insignificant). Then try to recognize common suffixes like -ng, -cuda, -opencl and -native. Finally make just a few hard-coded "fixes" (if needed). > We need to come up with a solution before we can release bleeding as the > next john-1.8 jumbo version. We don't *need* to, but it would be a good thing. I think we have plenty of time, with this pace (issues list growing instead of shrinking) I can't see a release in 2013. magnum
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.