Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 19 Aug 2013 21:07:20 +0200
From: Frank Dittrich <frank_dittrich@...mail.com>
To: john-dev@...ts.openwall.com
Subject: Re: relbench/benchmark-unify

On 08/19/2013 07:46 PM, magnum wrote:
> On 19 Aug, 2013, at 9:44 , Frank Dittrich <frank_dittrich@...mail.com> wrote:
>> My best idea so far is to add a new option to benchmark-unify:
>> --drop-format-labels[=0|=1]

I just implemented this now.

>> Default would be to just drop the format labels which are known to be
>> alternative implementations:
>> *-ng, *-naive, nt2:

I didn't implement this default behavior. I just implemented
--drop-format-labels[=0|=1].
Not specifying --drop-format-labels is the same as
--drop-format-labels=0, --drop-format-labels is the same as
--drop-format-labels=1.

Instead of removing labels based on name patterns, I added 3 mappings:
mschapv2-naive, MSCHAPv2 C/R	MSCHAPv2, C/R
netntlm-naive, NTLMv1 C/R	netntlm, NTLMv1 C/R
nt2, NT	NT

If --drop-labels is used., "netntlm, NTLMv1 C/R" gets converted to
"NTLMv1 C/R".


>> --drop-format-labels=0 would be to keep format labels even for those
>> alternative implementations.
>> --drop-format-labels or --drop-format-labels=1 would mean to drop all
>> format labels, even if this makes "RAdmin, v2.x" a useless format name

I could add conversion rules to change this into "RAdmin v2.x".
But I'd prefer to rethink the format names in a few cases.

I did, however add a single conversions as a short-term workaround:
Drupal7, $S$ (x16385)	Drupal 7, $S$ (x16385)
s/Drupal7/Drupal 7/ makes sure it is not detected as a label...

Other format names that might need to be reviewed if relbench -v output
should still print reasonable format names and if we want to avoid
different C/R formats being wrongly detected as different
implementations of the same algorithm (or if we want to be able to
compare CPU implementations and GPU implementations):

Blockchain, My Wallet (x10)
blockchain-opencl, blockchain My Wallet

Clipperz, SRP

Drupal7, $S$ (x16385)

Fortigate, FortiOS

IKE, PSK

MongoDB, system / network

Mozilla, key3.db

MSCHAPv2, C/R

OpenVMS, Purdy

PBKDF2-HMAC-SHA256-opencl, OpenCL
PBKDF2-HMAC-SHA256, rounds=12000
PBKDF2-HMAC-SHA512, GRUB2 / OS X 10.8

PFX, PKCS12 (.pfx, .p12)

PST, custom CRC-32

PuTTY, Private Key

RAdmin, v2.x

Raw-SHA, "SHA-0"

STRIP, Password Manager

WoWSRP, Battlenet


The GPU format name needs to change to WinZip:
zip-opencl, ZIP
ZIP, WinZip

These will never be detected as the same algorithm, because FORMAT_NAME
is empty so that only FORMAT_LABEL is printed:

Raw-SHA512-cuda [SHA512 CUDA (inefficient, development use mostly)]
Raw-SHA512-ng-i [SHA512 128/128 SSE4.1 2x]
Raw-SHA512-ng-opencl, (pwlen < 55) [SHA512 OpenCL (inefficient,
development use mostly)]
Raw-SHA512-ng [SHA512 128/128 SSSE3 2x]
Raw-SHA512-opencl [SHA512 OpenCL (inefficient, development use mostly)]
Raw-SHA512 [SHA512 128/128 SSE4.1 2x]


Is that right, that these are different formats (meaning we have a GPU
implementation without a CPU implementation):

ssha-opencl, Netscape LDAP {SSHA} [SHA1 OpenCL (inefficient, development
use mostly)]
nsldap, Netscape LDAP {SHA} [SHA1 128/128 AVX 4x]

(The test vectors seem to indicate they really differ. I wasn't aware of
formats without a CPU implementation.)


In this case, using passwords of different lengths for benchmarking is
unfortunate:
rar-opencl, RAR3 (6 characters)
rar, RAR3 (4 characters)



While we are at renaming formats: What about s/Office/MS Office/ here?

Office, 2007/2010 (SHA-1) / 2013 (SHA-512), with AES
office2007-opencl, Office 2007 (50,000 iterations)
office2010-opencl, Office 2010 (100,000 iterations)
office2013-opencl, Office 2013 (100,000 iterations)
oldoffice, Office <= 2003

And what about changing M$ to MS here:
mscash2-cuda, M$ Cache Hash 2 (DCC2)
mscash2, M$ Cache Hash 2 (DCC2)
mscash2-opencl, M$ Cache Hash 2 (DCC2)
mscash-cuda, M$ Cache Hash
mscash, M$ Cache Hash


Also, in the --test output we have one CRC-32 and one CRC32.

> I'm not sure. Perhaps another solution is another revision of the names and labels, after coming up with some convention(s).

I hope my comments made some sense, and Solar hadn't something
completely different in mind.

Instead of a patch, I'll attach my new benchmark-unify version.
(As an RFC - If you like the changes, feel free to commit - wuth or
without any adjustments you have in mind.)


Frank

#!/usr/bin/perl -w
#
# John the Ripper benchmark output conversion tool, revision 1.1
# Copyrigth (c) 2012-2013, Frank Dittrich
# Some code might be borrowed from the relbench script,
# Copyright (2) 2011 Solar Designer, because the code has
# originally been added as a patch to relbench.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted.  (This is a heavily cut-down "BSD license".)
#
# The script is used to unify the benchmark (./john --test) output
# of different John the Ripper versions (official or Jumbo)
# so that the format names match those used in the newest (Jumbo) version.
#
# This allows to use two files with benchmark output to be used
# with relbench.
#
# This Perl script reads "john --test" benchmark output from STDIN
# and writes the converted benchmark output to STDOUT.
#
# You can either use
#
# ./john --test > benchmark-orig.txt
#
# ./benchmark-unify < benchmark-orig.txt > benchmark-converted.txt
#
# Or, you can use
#
# ./john --test | ./benchmark-unify > benchmark-converted.txt
#
# in case you don't need the original file
#
# If you want to create a file with benchmark output and see the
# benchmark output on the screen while the benchmark runs, you can
# also use
#
# ./john --test | ./benchmark-unify | tee benchmark-converted.txt
#
# Two of those converted benchmark output files can be used with
# relbench, usually resulting in a higher number of matching
# format names which allow to compare the benchmark results
# of different benchmark runs.
#

sub parse
{
	chomp;
	($name,$end) = /^Benchmarking: (.*[^ ]) +(\[.*\].*)$/;
	if (defined($name) && defined($end)) {
		$name =~ s/(dynamic_[0-9]+):?.*/$1/;
		$name =~ s/\s+/ /g;
		$name =~ s/\[/(/;
		$name =~ s/\]/)/;

		if (defined($renamed{$name})) {
			$name = $renamed{$name};
		}
		if ($drop_labels == 1) {
			$name =~ s/^[^\s]*, (.*)/$1/;
		}
		print "Benchmarking: $name $end\n";
	}
	else {
		print "$_\n";
	}
}

sub usage
{
	die "\nUsage:\n$0 [--drop-labels[=0|=1]]\n\n($0 reads ./john --test output from stdin and writes to stdout.)\n\n"
}

if ($#ARGV == -1) {
	$drop_labels = -1;
}
elsif($#ARGV == 0) {
	if ($ARGV[0] eq "--drop-labels" || $ARGV[0] eq "--drop-labels=1" ) {
		$drop_labels = 1;
	}
	elsif ($ARGV[0] eq "--drop-labels=0" ) {
		$drop_labels = 0;
	}
	else {
print "$ARGV[0]\n";
		usage();
	}
}
else {
print "$#ARGV";
	usage();
}

$_ = '';

while(<DATA>) {
	chomp;
	($old_format, $new_format) = /^(.*)	(.*)$/;
	if(defined($new_format) && defined($old_format)) {
		# FIXME: Should I check that no format name appears
		#        both as an old name and as a new name?
		#        I don't want to map
		#        old -> new -> even newer
		#        or old -> new and new -> old
		#        (So far I just manually check for such problems
		#        whenever I change this file.)
		$renamed{$old_format} = $new_format;
	}
}

$_ = '';

while (<STDIN>) {
	parse();
}

# Mapping old format names to new ones, separated by \t
# old	new
# Currently not mapped john-1.7.9-jumbo-5 format names:
#
# HMAC MD5	HMAC MD5
# format name unchanged, but benchmark changed from Raw: to
# Only one salt: / Many salts:
#
# ssh	SSH RSA/DSA
# test vector has changed: (one 2048-bit RSA and one 1024-bit DSA key)
#
# For readability, please keep this list sorted by old format name
__DATA__
1Password Agile Keychain PBKDF2-HMAC-SHA-1 AES	agilekeychain, 1Password Agile Keychain
Apple DMG PBKDF2-HMAC-SHA-1 3DES / AES	dmg, Apple DMG
bf-opencl, OpenBSD Blowfish (x32)	bcrypt-opencl ("$2a$05", 32 iterations)
BLAKE2b 512	Raw-Blake2
BSDI DES (x725)	bsdicrypt, BSDI crypt(3) ("_J9..", 725 iterations)
Cisco PIX MD5	pix-md5, Cisco PIX
Clipperz SRP SHA256	Clipperz, SRP
CRC-32	CRC32
crypt-MD5	md5crypt, crypt(3) $1$
DIGEST-MD5	dmd5, DIGEST-MD5 C/R
DIGEST-MD5 C/R	dmd5, DIGEST-MD5 C/R
Django PBKDF2-HMAC-SHA-256 (x10000)	Django (x10000)
DragonFly BSD $3$ SHA-256 w/ bug, 32-bit	dragonfly3-32, DragonFly BSD $3$ w/ bug, 32-bit
DragonFly BSD $3$ SHA-256 w/ bug, 64-bit	dragonfly3-64, DragonFly BSD $3$ w/ bug, 64-bit
DragonFly BSD $4$ SHA-512 w/ bugs, 32-bit	dragonfly4-32, DragonFly BSD $4$ w/ bugs, 32-bit
DragonFly BSD $4$ SHA-512 w/ bugs, 64-bit	dragonfly4-64, DragonFly BSD $4$ w/ bugs, 64-bit
#FIXME: s/Drupal7,/Drupal 7,/
Drupal 7 $S$ SHA-512 (x16385)	Drupal 7, $S$ (x16385)
Drupal7, $S$ (x16385)	Drupal 7, $S$ (x16385)
#dynamic_20: Cisco PIX (MD5 salted)	dynamic_20: Cisco ASA (MD5 salted)
#dynamic_38: sha1($s.sha1($s.($p))) (Wolt3BB)	dynamic_38: sha1($s.sha1($s.sha1($p))) (Wolt3BB)
Eggdrop	bfegg, Eggdrop
Eggdrop Blowfish	bfegg, Eggdrop
EncFS PBKDF2 AES / Blowfish	EncFS
EPiServer SID	EPI, EPiServer SID
EPiServer SID Hashes	EPI, EPiServer SID
EPiServer SID salted SHA-1	EPI, EPiServer SID
EPiServer salted SHA-1/SHA-256	EPiServer
FormSpring sha256($salt.$pass)	FormSpring
Fortigate FortiOS	Fortigate, FortiOS
FreeBSD MD5	md5crypt, crypt(3) $1$
generic crypt(3)	crypt, generic crypt(3) DES
generic crypt(3) DES	crypt, generic crypt(3) DES
Generic salted MD4	md4-gen, Generic salted MD4
#FIXME: Generic + non-generic?
Generic salted SHA-1	sha1-gen, Generic salted SHA-1
GNOME Keyring iterated-SHA256 AES	keyring, GNOME Keyring
GOST R 34.11-94	gost, GOST R 34.11-94
HalfLM C/R DES	nethalflm, HalfLM C/R
HMAC MD5	HMAC-MD5
HMAC SHA-1	HMAC-SHA1
HMAC SHA-224	HMAC-SHA224
HMAC SHA-256	HMAC-SHA256
HMAC SHA-384	HMAC-SHA384
HMAC SHA-512	HMAC-SHA512
hmailserver	hMailServer
hMailServer salted SHA-256	hMailServer
HTTP Digest access authentication	hdaa, HTTP Digest access authentication
HTTP Digest access authentication MD5	hdaa, HTTP Digest access authentication
IKE PSK HMAC-MD5 / HMAC-SHA1	IKE, PSK
Invision Power Board 2.x salted MD5	ipb2, Invision Power Board 2.x
IPB2 MD5	ipb2, Invision Power Board 2.x
iSCSI CHAP authentication MD5	chap, iSCSI CHAP authentication
KDE KWallet SHA-1	kwallet, KDE KWallet
KeePass SHA-256 AES	KeePass
Kerberos 5 db etype 18 aes256-cts-hmac-sha1-96	krb5-18, Kerberos 5 db etype 18 aes256-cts-hmac-sha1-96
Kerberos 5 db etype 23 rc4-hmac	krb5-23, Kerberos 5 db etype 23 rc4-hmac
Kerberos 5 AS-REQ Pre-Auth etype 17/18 aes-cts-hmac-sha1-96	krb5pa-sha1, Kerberos 5 AS-REQ Pre-Auth etype 17/18 aes-cts-hmac-sha1-96
Kerberos 5 AS-REQ Pre-Auth etype 23 md4, rc4-hmac-md5	krb5pa-md5, Kerberos 5 AS-REQ Pre-Auth etype 23 md4, rc4-hmac-md5
Kerberos AFS DES	AFS, Kerberos AFS
Kerberos v4 TGT	krb4, Kerberos v4 TGT
Kerberos v4 TGT DES	krb4, Kerberos v4 TGT
Kerberos v5 TGT	krb5, Kerberos v5 TGT
Kerberos v5 TGT 3DES	krb5, Kerberos v5 TGT
KRB5 aes256-cts-hmac-sha1-96	krb5-18, Kerberos 5 db etype 18 aes256-cts-hmac-sha1-96
KRB5 arcfour-hmac	krb5-23, Kerberos 5 db etype 23 rc4-hmac
LastPass offline PBKDF2 SHA1	LastPass offline
#FIXME: ,
LastPass sniffed sessions PBKDF2-HMAC-SHA-256 AES	LastPass, sniffed sessions
LM C/R DES	netlm, LM C/R
LM DES	LM
LMv2 C/R MD4 HMAC-MD5	netlmv2, LMv2 C/R
Lotus Notes/Domino 5	lotus5, Lotus Notes/Domino 5
Lotus Notes/Domino 6 More Secure Internet Password	dominosec, Lotus Notes/Domino 6 More Secure Internet Password
Lotus5	lotus5, Lotus Notes/Domino 5
Mac OS X Keychain PBKDF2-HMAC-SHA-1 3DES	keychain, Mac OS X Keychain
Mac OS X 10.4 - 10.6 salted SHA-1	xsha, Mac OS X 10.4 - 10.6
Mac OS X 10.7+ salted SHA-512	xsha512, Mac OS X 10.7+
md5(unicode($p))	Raw-MD5u
MediaWiki -- md5($s.'-'.md5($p))	MediaWiki
MediaWiki md5($s.'-'.md5($p))	MediaWiki
MongoDB system / network MD5	MongoDB, system / network
More Secure Internet Password	dominosec, Lotus Notes/Domino 6 More Secure Internet Password
Mozilla SHA-1 3DES	Mozilla, key3.db
Mozilla (key3.db) SHA-1 3DES	Mozilla, key3.db
M$ Cache Hash	mscash, M$ Cache Hash
M$ Cache Hash MD4	mscash, M$ Cache Hash
M$ Cache Hash 2 (DCC2)	mscash2, M$ Cache Hash 2 (DCC2)
M$ Cache Hash 2 (DCC2) PBKDF2-HMAC-SHA-1	mscash2, M$ Cache Hash 2 (DCC2)
MS Kerberos 5 AS-REQ Pre-Auth	krb5pa-md5, Kerberos 5 AS-REQ Pre-Auth etype 23 md4, rc4-hmac-md5
MS Kerberos 5 AS-REQ Pre-Auth MD4 MD5 RC4	krb5pa-md5, Kerberos 5 AS-REQ Pre-Auth etype 23 md4, rc4-hmac-md5
MS-SQL	mssql, MS SQL
MS SQL SHA-1	mssql, MS SQL
MS-SQL05	mssql05, MS SQL 2005
MS SQL 2005 SHA-1	mssql05, MS SQL 2005
MS SQL 2012 SHA512	mssql12, MS SQL 2012
MSCHAPv2 C/R MD4 DES	MSCHAPv2, C/R
mschapv2-naive, MSCHAPv2 C/R	MSCHAPv2, C/R
MYSQL	mysql, MySQL
MySQL	mysql, MySQL
MYSQL_fast	mysql, MySQL
MySQL 4.1 double-SHA-1	mysql-sha1, MySQL 4.1
MySQL Network Authentication SHA1	mysqlna, MySQL Network Authentication
netntlm-naive, NTLMv1 C/R	netntlm, NTLMv1 C/R
Netscape LDAP SHA	nsldap, Netscape LDAP {SHA}
Netscape LDAP SHA-1	nsldap, Netscape LDAP {SHA}
Netscreen MD5	md5ns, Netscreen
NT v2	NT
NT MD4	NT
nt2, NT	NT
NTLMv1 C/R MD4 DES	netntlm, NTLMv1 C/R
NTLMv2 C/R MD4 HMAC-MD5	netntlmv2, NTLMv2 C/R
Nuked-Klan CMS SHA1 MD5	nk, Nuked-Klan CMS
ODF SHA-1 Blowfish	ODF
ODF SHA-1 Blowfish / SHA-256 AES	ODF
Office <= 2003 MD5/SHA-1, RC4	oldoffice, Office <= 2003
Office 2007/2010 SHA-1/AES	Office, 2007/2010 (SHA-1) / 2013 (SHA-512), with AES
Office 2007/2010 (SHA-1) / 2013 (SHA-512), with AES	Office, 2007/2010 (SHA-1) / 2013 (SHA-512), with AES
OpenBSD Blowfish (x32)	bcrypt ("$2a$05", 32 iterations)
OpenPGP / GnuPG Secret Key	gpg, OpenPGP / GnuPG Secret Key
OpenVMS Purdy	OpenVMS, Purdy
Oracle	oracle, Oracle 10
Oracle 10 DES	oracle, Oracle 10
Oracle 11g	oracle11, Oracle 11g
Oracle 11g SHA-1	oracle11, Oracle 11g
Oracle O5LOGON protocol	o5logon, Oracle O5LOGON protocol
osCommerce md5($salt.$pass)	osc, osCommerce
Password Safe SHA-256	pwsafe, Password Safe
PBKDF2-HMAC-SHA512 GRUB2 / OS X 10.8	PBKDF2-HMAC-SHA512, GRUB2 / OS X 10.8
pdf	PDF
PDF MD5 RC4	PDF
PDF MD5 SHA-2 RC4 / AES	PDF
#FIXME: phpass ($P$9 or $H$9)
PHPass MD5	phpass ($P$9)
phpass MD5 ($P$9)	phpass ($P$9)
PHPS -- md5(md5($pass).$salt)	PHPS
PHPS md5(md5($pass).$salt)	PHPS
PIX MD5	pix-md5, Cisco PIX
PKCS12 (.pfx, .p12)	PFX, PKCS12 (.pfx, .p12)
pkzip	PKZIP
Post.Office MD5	po, Post.Office
PostgreSQL MD5 challenge-response	postgres, PostgreSQL C/R
#FIXME CRC-32 vs. CRC32
PST custom CRC-32	PST, custom CRC-32
PuTTY Private Key SHA-1 / AES	PuTTY, Private Key
RACF DES	RACF
RAdmin v2.x MD5	RAdmin, v2.x
rar	rar, RAR3 (4 characters)
RAR3 SHA-1 AES (4 characters)	rar, RAR3 (4 characters)
Raw MD4	Raw-MD4
Raw MD5	Raw-MD5
Raw SHA	Raw-SHA, "SHA-0"
Raw SHA-0	Raw-SHA, "SHA-0"
Raw SHA-1	Raw-SHA1
Raw SHA-1 (pwlen <= 15)	Raw-SHA1-ng, (pwlen <= 15)
#FIXME s/Linkedin/LinkedIn/
Raw SHA-1 LinkedIn	Raw-SHA1-Linkedin
Raw SHA-224	Raw-SHA224
Raw SHA-256	Raw-SHA256
Raw SHA-384	Raw-SHA384
Raw SHA-512	Raw-SHA512
Salted SHA-1	Salted-SHA1
SAP BCODE	sapb, SAP CODVN B (BCODE)
SAP CODVN B (BCODE)	sapb, SAP CODVN B (BCODE)
SAP CODVN G (PASSCODE)	sapg, SAP CODVN F/G (PASSCODE)
SAP CODVN F/G (PASSCODE)	sapg, SAP CODVN F/G (PASSCODE)
sha256crypt (rounds=5000)	sha256crypt, crypt(3) $5$ (rounds=5000)
sha512crypt (rounds=5000)	sha512crypt, crypt(3) $6$ (rounds=5000)
Siemens S7 HMAC-SHA-1	Siemens-S7
SIP MD5	SIP
SSH RSA/DSA (one 2048-bit RSA and one 1024-bit DSA key)	SSH (one 2048-bit RSA and one 1024-bit DSA key)
ssh-ng SSH RSA / DSA	SSH-ng
StarOffice SXC SHA-1 Blowfish	sxc, StarOffice .sxc
STRIP Password Manager	STRIP, Password Manager
sybasease	sybasease, Sybase ASE
Sybase ASE salted SHA-256	sybasease, Sybase ASE
Traditional DES	descrypt, traditional crypt(3)
Tripcode DES	tripcode
TrueCrypt RIPEMD160 AES256_XTS	tc_ripemd160, TrueCrypt RIPEMD160 AES256_XTS
TrueCrypt SHA512 AES256_XTS	tc_sha512, TrueCrypt SHA512 AES256_XTS
TrueCrypt WHIRLPOOL AES256_XTS	tc_whirlpool, TrueCrypt WHIRLPOOL
VNC DES	VNC
WoltLab BB3 salted SHA-1	wbb3, WoltLab BB3
WoW (Battlenet) SRP sha1	WoWSRP, Battlenet
WPA-PSK PBKDF2-HMAC-SHA-1	WPAPSK
zip	ZIP, WinZip
WinZip PBKDF2-HMAC-SHA-1	ZIP, WinZip

Powered by blists - more mailing lists

Your e-mail address:

Powered by Openwall GNU/*/Linux - Powered by OpenVZ