Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 19 Aug 2013 21:07:20 +0200
From: Frank Dittrich <>
Subject: Re: relbench/benchmark-unify

On 08/19/2013 07:46 PM, magnum wrote:
> On 19 Aug, 2013, at 9:44 , Frank Dittrich <> wrote:
>> My best idea so far is to add a new option to benchmark-unify:
>> --drop-format-labels[=0|=1]

I just implemented this now.

>> Default would be to just drop the format labels which are known to be
>> alternative implementations:
>> *-ng, *-naive, nt2:

I didn't implement this default behavior. I just implemented
Not specifying --drop-format-labels is the same as
--drop-format-labels=0, --drop-format-labels is the same as

Instead of removing labels based on name patterns, I added 3 mappings:
mschapv2-naive, MSCHAPv2 C/R	MSCHAPv2, C/R
netntlm-naive, NTLMv1 C/R	netntlm, NTLMv1 C/R
nt2, NT	NT

If --drop-labels is used., "netntlm, NTLMv1 C/R" gets converted to
"NTLMv1 C/R".

>> --drop-format-labels=0 would be to keep format labels even for those
>> alternative implementations.
>> --drop-format-labels or --drop-format-labels=1 would mean to drop all
>> format labels, even if this makes "RAdmin, v2.x" a useless format name

I could add conversion rules to change this into "RAdmin v2.x".
But I'd prefer to rethink the format names in a few cases.

I did, however add a single conversions as a short-term workaround:
Drupal7, $S$ (x16385)	Drupal 7, $S$ (x16385)
s/Drupal7/Drupal 7/ makes sure it is not detected as a label...

Other format names that might need to be reviewed if relbench -v output
should still print reasonable format names and if we want to avoid
different C/R formats being wrongly detected as different
implementations of the same algorithm (or if we want to be able to
compare CPU implementations and GPU implementations):

Blockchain, My Wallet (x10)
blockchain-opencl, blockchain My Wallet

Clipperz, SRP

Drupal7, $S$ (x16385)

Fortigate, FortiOS


MongoDB, system / network

Mozilla, key3.db


OpenVMS, Purdy

PBKDF2-HMAC-SHA256-opencl, OpenCL
PBKDF2-HMAC-SHA256, rounds=12000

PFX, PKCS12 (.pfx, .p12)

PST, custom CRC-32

PuTTY, Private Key

RAdmin, v2.x

Raw-SHA, "SHA-0"

STRIP, Password Manager

WoWSRP, Battlenet

The GPU format name needs to change to WinZip:
zip-opencl, ZIP
ZIP, WinZip

These will never be detected as the same algorithm, because FORMAT_NAME
is empty so that only FORMAT_LABEL is printed:

Raw-SHA512-cuda [SHA512 CUDA (inefficient, development use mostly)]
Raw-SHA512-ng-i [SHA512 128/128 SSE4.1 2x]
Raw-SHA512-ng-opencl, (pwlen < 55) [SHA512 OpenCL (inefficient,
development use mostly)]
Raw-SHA512-ng [SHA512 128/128 SSSE3 2x]
Raw-SHA512-opencl [SHA512 OpenCL (inefficient, development use mostly)]
Raw-SHA512 [SHA512 128/128 SSE4.1 2x]

Is that right, that these are different formats (meaning we have a GPU
implementation without a CPU implementation):

ssha-opencl, Netscape LDAP {SSHA} [SHA1 OpenCL (inefficient, development
use mostly)]
nsldap, Netscape LDAP {SHA} [SHA1 128/128 AVX 4x]

(The test vectors seem to indicate they really differ. I wasn't aware of
formats without a CPU implementation.)

In this case, using passwords of different lengths for benchmarking is
rar-opencl, RAR3 (6 characters)
rar, RAR3 (4 characters)

While we are at renaming formats: What about s/Office/MS Office/ here?

Office, 2007/2010 (SHA-1) / 2013 (SHA-512), with AES
office2007-opencl, Office 2007 (50,000 iterations)
office2010-opencl, Office 2010 (100,000 iterations)
office2013-opencl, Office 2013 (100,000 iterations)
oldoffice, Office <= 2003

And what about changing M$ to MS here:
mscash2-cuda, M$ Cache Hash 2 (DCC2)
mscash2, M$ Cache Hash 2 (DCC2)
mscash2-opencl, M$ Cache Hash 2 (DCC2)
mscash-cuda, M$ Cache Hash
mscash, M$ Cache Hash

Also, in the --test output we have one CRC-32 and one CRC32.

> I'm not sure. Perhaps another solution is another revision of the names and labels, after coming up with some convention(s).

I hope my comments made some sense, and Solar hadn't something
completely different in mind.

Instead of a patch, I'll attach my new benchmark-unify version.
(As an RFC - If you like the changes, feel free to commit - wuth or
without any adjustments you have in mind.)


View attachment "benchmark-unify" of type "text/plain" (11599 bytes)

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.