[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Fri, 25 Oct 2013 00:58:50 +0200
From: magnum <john.magnum@...hmail.com>
To: john-dev@...ts.openwall.com
Subject: Re: benchmark-unify and relbench adjustments
On 2013-10-24 14:01, Frank Dittrich wrote:
> magnum, Solar, all,
>
> changed format names and changed --test output require some adjustments
> to increase the number of benchmarks which can be compared (between
> bleeding and earlier versions, e.g., unstable.
> Finally, I don't know yet how to handle this change:
>
> In addition to the format name, the format label has been included into
> the output, at least if format label and format name differ.
Format label is no longer all-lowercase and is always printed. Format
name can now be empty but is printed if it isn't.
> I was experimenting with another solution (this patch just checks an
> environment variable instead of a new benchmark-unify option, but I can
> provide a patch which introduces a --drop-labels option):
> This will increase the number of benchmarks in bleeding that can be
> compared with benchmarks in unstable, but it has unwanted side effects:
>
> -the user would need to run benchmark-unify even on the --test output of
> the latest john version (so far I always aimed at converting older
> output to the newest, and keeping the newest output unchanged)
>
> -in some cases, the format name will become (completely) meaningless
> and/or ambiguous:
Wouldn't the opposite be safer and easier: Only care about format label
(anything up to first comma or bracket, and case insignificant). Then
try to recognize common suffixes like -ng, -cuda, -opencl and -native.
Finally make just a few hard-coded "fixes" (if needed).
> We need to come up with a solution before we can release bleeding as the
> next john-1.8 jumbo version.
We don't *need* to, but it would be a good thing. I think we have plenty
of time, with this pace (issues list growing instead of shrinking) I
can't see a release in 2013.
magnum
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
