Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 19 Aug 2013 09:44:27 +0200
From: Frank Dittrich <>
Subject: Re: relbench/benchmark-unify

On 08/19/2013 08:59 AM, Frank Dittrich wrote:
> If I have a string like "mschapv2-naive, MSCHAPv2 C/R", should I always
> drop the "format-label, " part in benchmark-unify, so that I still
> detect multiple implementations of the same algorithm, picking the fastest?
> But then, this would mean that "descrypt, traditional crypt(3)" would
> become "traditional crypt(3)".
> And "md5crypt, crypt(3) $1$" would become "crypt(3) $1$".
> I'm afraid there are some cases where important information gets lost:
> LastPass, sniffed sessions
> MongoDB, system / network
> PST, custom CRC-32
> PuTTY, Private Key
> RAdmin, v2.x
> The other approach would be to keep all those format labels, and just
> drop those few labels where I have different implementations of the same
> algorithms, allowing the fastest one to be picked for comparison.
> But that would mean even more future benchmark-unify changes if
> additional implementations for existing algorithms come up.

Checking CUDA and OpenCL formats on bull, I noticed one more problem.

Benchmarking: md5crypt [CUDA]... DONE
Benchmarking: md5crypt [OpenCL]... DONE
Benchmarking: md5crypt-cuda, crypt(3) $1$ [MD5 CUDA]... DONE
Benchmarking: md5crypt-opencl, crypt(3) $1$ [MD5 OpenCL]... DONE

May be it doesn't make much sense to compare CPU and GPU formats anyway.
On the other hand, measuring how much faster a GPU implementation is
compared to a CPU implementation for a given format might be useful
And, without dropping the format labels, I won't be able to easily
compare CUDA and OPenCL implementations.

This is getting ugly.
Ideas how to best handle the situation are most welcome.

My best idea so far is to add a new option to benchmark-unify:

Default would be to just drop the format labels which are known to be
alternative implementations:
*-ng, *-naive, nt2:

--drop-format-labels=0 would be to keep format labels even for those
alternative implementations.
--drop-format-labels or --drop-format-labels=1 would mean to drop all
format labels, even if this makes "RAdmin, v2.x" a useless format name

May be we need to fix those few format names which get meaningless
without the format label.

Any thoughts?


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.