Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sun, 24 Jun 2012 08:55:18 +0200
From: Frank Dittrich <frank_dittrich@...mail.com>
To: john-dev@...ts.openwall.com
CC: Dhiru Kholia <dhiru.kholia@...il.com>
Subject: Re: Several enhancements for relbench

On 06/24/2012 06:40 AM, Dhiru Kholia wrote:
> I haven't applied your patches since I don't understand them ;). Can
> you send all the patches to apply in a single mail (in git patch
> format)? Thanks.

OK, a single mail, but several patches (in git patch format) as they
were posted earlier.

The most important now are (to apply in this sequence):

0001-benchmark-unify-a-Perl-script-to-convert-benchmark-o.patch
from my mail
Re: [john-dev] relbench: map old and new format names
06/23/2012 01:00 AM
(An earlier version of that patch was "rejected" by Alexander because
the functionality was implemented directly in relbench instead of in a
separate script.)

benchmark-unify will convert the output of

./john --test

in a way that the format names used by an older john version will be
converted into the format name used by the latest version.

This allows

./relbench file1 file2

to find more matching formats, so that more benchmarks can be compared
than without such a format conversion.


On top of that one (not because there would be any collisions, but
because a changed output of the relbench script refers to the
benchmark-unify script created with the patch mentioned above.

0001-Several-enhancements-for-relbench.patch
from my mail
[john-dev] Several enhancements for relbench
06/23/2012 07:17 PM

(Please see my mail / the complete thread discussing the change with
Alexander) for details.

These two should apply without problems, because noone else is working
on those files.



Less urgent now, but important to apply before a jumbo-6 release,
because I change the output of --list=format-details, inserting a new
column in the middle.
(Such a change is only possible because --list=format-details is a new
feature, not one that is already in use by someone.)

0001-list-format-details-add-number-of-test-cases-to-outp.patch
from my mail
[john-dev] Add number of hard coded test cases to --list=format-details
output
06/23/2012 01:35 PM

This patch ads the number of format test cases hard coded in the source
code to the --list=format-details output.

And this one on top, because it fixes gcc warnings about unused
variables introduced by
0001-list-format-details-add-number-of-test-cases-to-outp.patch:

0001-Fix-gcc-warnings-in-list-format-details-code.patch
from my mail
Re: [john-dev] Add number of hard coded test cases to
--list=format-details output
06/23/2012 08:57 PM

These two should also apply without problems.
The line numbers might have changed.
If there is a problem applying those now, I'll submit a new patch
instead. The first two patches are more important.


All those patches have been created using

	git format-patch HEAD -1

after committing the changes to my local test branch.
Usually I add a good descrition during the commit.


Frank

>>From 730f1a0b9209fad8a25bd2ca79b0f72222b271ff Mon Sep 17 00:00:00 2001
From: Frank Dittrich <frank_dittrich@...mail.com>
Date: Sat, 23 Jun 2012 00:51:58 +0200
Subject: [PATCH] benchmark-unify: a Perl script to convert benchmark output
 for relbench

---
 run/benchmark-unify |  140 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 140 insertions(+), 0 deletions(-)
 create mode 100755 run/benchmark-unify

diff --git a/run/benchmark-unify b/run/benchmark-unify
new file mode 100755
index 0000000..8a19972
--- /dev/null
+++ b/run/benchmark-unify
@@ -0,0 +1,140 @@
+#!/usr/bin/perl -w
+#
+# John the Ripper benchmark output conversion tool, revision 1
+# Copyrigth (c) 2012, Frank Dittrich
+# Some code might be borrowed from the relbench script,
+# Copyright (2) 2011 Solar Designer, because the code has
+# originally been added as a patch to relbench.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted.  (This is a heavily cut-down "BSD license".)
+#
+# The script is used to unify the benchmark (./john --test) output
+# of different John the Ripper versions (official or Jumbo)
+# so that the format names match those used in the newest (Jumbo) version.
+#
+# This allows to use two files with benchmark output to be used
+# with relbench. 
+#
+# This Perl script reads "john --test" benchmark output from STDIN
+# and writes the converted benchmark output to STDOUT.
+#
+# You can either use
+#
+# ./john --test > benchmark-orig.txt
+#
+# ./benchmark-unify < benchmark-orig.txt > benchmark-converted.txt
+#
+# Or, you can use
+#
+# ./john --test | ./benchmark-unify > benchmark-converted.txt
+#
+# in case you don't need the original file
+#
+# If you want to create a file with benchmark output and see the
+# benchmark output on the screen while the benchmark runs, you can
+# also use
+#
+# ./john --test | ./benchmark-unify | tee benchmark-converted.txt
+#
+# Two of those converted benchmark output files can be used with
+# relbench, usually resulting in a higher number of matching
+# format names which allow to compare the benchmark results
+# of different benchmark runs.
+#
+
+sub parse
+{
+	chomp;
+	($name,$end) = /^Benchmarking: (.*[^ ]) +(\[.*\].*)$/;
+	if (defined($name) && defined($end)) {
+		$name =~ s/\s+/ /g;
+		$name =~ s/\[/(/;
+		$name =~ s/\]/)/;
+
+		if (defined($renamed{$name})) {
+			$name = $renamed{$name};
+		}
+		print "Benchmarking: $name $end\n";
+	}
+	else {
+		print "$_\n";
+	}
+}
+
+$_ = '';
+
+while(<DATA>) {
+	chomp;
+	($old_format, $new_format) = /^(.*)	(.*)$/;
+	if(defined($new_format) && defined($old_format)) {
+		# FIXME: Should I check that no format name appears
+		#        both as an old name and as a new name?
+		#        I don't want to map
+		#        old -> new -> even newer
+		#        or old -> new and new -> old
+		$renamed{$old_format} = $new_format;
+	}
+}
+
+$_ = '';
+
+while (<>) {
+	parse();
+}
+
+# Mapping old format names to new ones, separated by \t
+# old	new
+# Currently not mapped john-1.7.9-jumbo-5 format names:
+#
+# HMAC MD5	HMAC MD5
+# format name unchanged, but benchmark changed from Raw: to
+# Only one salt: / Many salts:
+#
+# MYSQL	MySQL
+# 2 different versions for john-1.7.9-jumbo-5, for next jumbo
+# the MYSQL_fast survived as MySQL, the MYSQL got moved to unused
+#
+# NT	NT (label nt)
+# format name not yet changed
+#
+# NT v2	NT (label nt2)
+# format name not yet changed
+#
+#
+# ssh	SSH RSA/DSA
+# test vector has changed: (one 2048-bit RSA and one 1024-bit DSA key)
+#
+# For readability, please keep this list sorted by old format name
+__DATA__
+DIGEST-MD5	DIGEST-MD5 C/R
+EPiServer SID Hashes	EPiServer SID salted SHA-1
+Eggdrop	Eggdrop Blowfish
+HTTP Digest access authentication	HTTP Digest access authentication MD5
+IPB2 MD5	Invision Power Board 2.x salted MD5
+Kerberos v4 TGT	Kerberos v4 TGT DES
+Kerberos v5 TGT	Kerberos v5 TGT 3DES
+Lotus5	Lotus Notes/Domino 5
+M$ Cache Hash	M$ Cache Hash MD4
+M$ Cache Hash 2 (DCC2)	M$ Cache Hash 2 (DCC2) PBKDF2-HMAC-SHA-1
+MS Kerberos 5 AS-REQ Pre-Auth	MS Kerberos 5 AS-REQ Pre-Auth MD4 MD5 RC4
+MS-SQL	MS SQL SHA-1
+MS-SQL05	MS SQL 2005 SHA-1
+MYSQL_fast	MySQL
+MediaWiki -- md5($s.'-'.md5($p))	MediaWiki md5($s.'-'.md5($p))
+More Secure Internet Password	Lotus Notes/Domino 6 More Secure Internet Password
+Netscape LDAP SHA	Netscape LDAP SHA-1
+Oracle	Oracle 10 DES
+Oracle 11g	Oracle 11g SHA-1
+PHPS -- md5(md5($pass).$salt)	PHPS md5(md5($pass).$salt)
+PHPass MD5	phpass MD5 ($P$9)
+Raw SHA	Raw SHA-0
+SAP BCODE	SAP CODVN B (BCODE)
+SAP CODVN G (PASSCODE)	SAP CODVN F/G (PASSCODE)
+generic crypt(3)	generic crypt(3) DES
+hmailserver	hMailServer salted SHA-256
+pdf	PDF MD5 RC4
+pkzip	PKZIP
+rar	RAR3 SHA-1 AES (4 characters)
+sybasease	Sybase ASE salted SHA-256
+zip	WinZip PBKDF2-HMAC-SHA-1
-- 
1.7.7.6


>>From effacd2ec1fe22db7d4af72c0b35bb08710b47ba Mon Sep 17 00:00:00 2001
From: Frank Dittrich <frank_dittrich@...mail.com>
Date: Sat, 23 Jun 2012 20:54:49 +0200
Subject: [PATCH] Fix gcc warnings in --list=format-details code

---
 src/john.c |    7 +------
 1 files changed, 1 insertions(+), 6 deletions(-)

diff --git a/src/john.c b/src/john.c
index 97057db..88d0af5 100644
--- a/src/john.c
+++ b/src/john.c
@@ -770,14 +770,9 @@ static void john_init(char *name, int argc, char **argv)
 	if (options.listconf &&
 	    !strcasecmp(options.listconf, "format-details")) {
 		struct fmt_main *format;
-		int i;
-
-		i = 0;
 		format = fmt_list;
 		do {
-			int ntests;
-			struct fmt_tests *current;
-			ntests = 0;
+			int ntests = 0;
 			
 			if(format->params.tests) {
 				while (format->params.tests[ntests++].ciphertext);
-- 
1.7.7.6


>>From 710dd37269045b54ae8666c5cd439190f27fa5a2 Mon Sep 17 00:00:00 2001
From: Frank Dittrich <frank_dittrich@...mail.com>
Date: Sat, 23 Jun 2012 13:26:15 +0200
Subject: [PATCH] --list=format-details: add number of test cases to output

Furthermore, clanup unnecessary code for --list=format-details
(which had been copied from the --list=formats code)

Added --list=formats and --list=format-details description to
doc/OPTIONS
---
 doc/OPTIONS |   21 +++++++++++++++++++++
 src/john.c  |   16 ++++++++++------
 2 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/doc/OPTIONS b/doc/OPTIONS
index d455f5b..b501bb0 100644
--- a/doc/OPTIONS
+++ b/doc/OPTIONS
@@ -289,6 +289,27 @@ The different variants of list externals may be worth mentioning:
 --list=ext-modes        list external modes [has a generate()]
 --list=ext-filters      list external filters, including modes with a filter
 --list=ext-filters-only list external filters [lacks a generate()]
+--list=formats          list all the supported formats in the sequence they
+                        are registered (this is also the sequence that will
+                        be used to identify the first format which detects
+                        a valid ciphertext (password hash) in the input file 
+                        if john is started without the --format=NAME option)
+--list=format-details   list all the suported formats in the same sequence as
+                        with --list=formats, but with additional details
+                        -format label
+                         this is to be used as NAME in --format=NAME
+                        -maximum supported password length in bytes
+                         for performance reasond, the maximum length
+                         supported by John the Ripper can be shorter than
+                         the maximum length supported by the application
+                         which uses this format
+                        -minimum keys per crypt
+                        -maximum keys per crypt
+                        -format flags
+                        -number of hard coded test cases for --test
+                        -algorithm name
+                        -format name
+                        (the individual columns are separated by [tab])
 
 
 --regen-lost-salts=N	Try to find password AND salt in a set of raw hashes.
diff --git a/src/john.c b/src/john.c
index ec07aba..97057db 100644
--- a/src/john.c
+++ b/src/john.c
@@ -774,23 +774,27 @@ static void john_init(char *name, int argc, char **argv)
 
 		i = 0;
 		format = fmt_list;
-		while ((format = format->next))
-			i++;
-
-		i = 0;
-		format = fmt_list;
 		do {
+			int ntests;
+			struct fmt_tests *current;
+			ntests = 0;
+			
+			if(format->params.tests) {
+				while (format->params.tests[ntests++].ciphertext);
+				ntests--;
+			}
 			/*
 			 * FIXME: Are other parameters more important?
 			 *        Should I use hexadecimal output
 			 *        for the FMT_flags?
 			 */
-			printf("%s\t%d\t%d\t%d\t%d\t%s\t%s\n",
+			printf("%s\t%d\t%d\t%d\t%d\t%d\t%s\t%s\n",
 			       format->params.label,
 			       format->params.plaintext_length,
 			       format->params.min_keys_per_crypt,
 			       format->params.max_keys_per_crypt,
 			       format->params.flags,
+			       ntests,
 			       format->params.algorithm_name,
 			       format->params.format_name);
 		} while ((format = format->next));
-- 
1.7.7.6


>>From a5255321417e1d453908e1c17edbcc9c15109f65 Mon Sep 17 00:00:00 2001
From: Frank Dittrich <frank_dittrich@...mail.com>
Date: Sat, 23 Jun 2012 18:51:32 +0200
Subject: [PATCH] Several enhancements for relbench

1.
If there are benchmarks that exist only in file 1 and benchmarks
that exist only in file 2, point out the possibility to convert
both benchmark files using benchmark-unify, so that more
benchmarks might be compared

2.
If a particular benchmark appears several times in the same file,
print a message to STDERR, and pick the higher values for
comparision (higher real value is considered here, only if the
real values are the same, the higher virtual value is considered).

3.
Add support for an optional verbose mode:

./relbench -v file1 file2
instead of
./relbench file1 file2

With verbose mode, for each benchmark that exists in both files,
the ratio of real / virtual c/s rate is printed like this:

Ratio:	0.95543 real, 0.96987 virtual	generic crypt(3) DES:Only one salt
Ratio:	1.00379 real, 1.00388 virtual	HalfLM C/R DES:Many salts
Ratio:	1.06590 real, 1.06586 virtual	FreeBSD MD5:Raw
Ratio:	2.92454 real, 1.47022 virtual	Invision Power Board 2.x salted MD5:Only one salt
Ratio:	0.97287 real, 0.97287 virtual	dynamic_20: Cisco PIX (MD5 salted):Many salts

This allows to use

./relbench -v file1 file2 |grep "^Ratio:"|cut -f 2-|sort -nr

This way, the benchmarks with the best performance improvement and the benchmarks
with the worst performance regression can easily be identified.
---
 run/relbench |   55 ++++++++++++++++++++++++++++++++++++++++++++++---------
 1 files changed, 46 insertions(+), 9 deletions(-)

diff --git a/run/relbench b/run/relbench
index 022d54c..e2f5ccf 100755
--- a/run/relbench
+++ b/run/relbench
@@ -26,6 +26,8 @@
 #
 
 $warned = 0;
+$onlyin1 = 0;
+$onlyin2 = 0;
 
 sub parse
 {
@@ -63,19 +65,33 @@ sub parse
 	print STDERR "Could not parse: $_\n" if (!$ok);
 }
 
-die "Usage: $0 BENCHMARK-FILE-1 BENCHMARK-FILE-2\n" if ($#ARGV != 1);
+die "Usage: $0 [-v] BENCHMARK-FILE-1 BENCHMARK-FILE-2\n" if ($#ARGV != 1 && ($#ARGV != 2 || $ARGV[0] ne '-v'));
 
-open(B1, '<' . $ARGV[0]) || die "Could not open file: $ARGV[0] ($!)";
-open(B2, '<' . $ARGV[1]) || die "Could not open file: $ARGV[1] ($!)";
+if ($#ARGV != 1) { 
+	open(B1, '<' . $ARGV[1]) || die "Could not open file: $ARGV[1] ($!)";
+	open(B2, '<' . $ARGV[2]) || die "Could not open file: $ARGV[2] ($!)";
+	$verbose = 1;
+} else {
+	open(B1, '<' . $ARGV[0]) || die "Could not open file: $ARGV[0] ($!)";
+	open(B2, '<' . $ARGV[1]) || die "Could not open file: $ARGV[1] ($!)";
+	$verbose = 0;
+}
 
 $_ = '';
 parse();
 while (<B1>) {
 	parse();
 	next unless (defined($id));
-	$b1r{$id} = $real;
-	$b1v{$id} = $virtual;
-}
+	if(defined($b2r{$id})) {
+		print STDERR "More than one benchmark for $id in file 2\n";
+		if($real > $b1r{$id} || ($real == $b1r{$id} && $virtual > $b1v{$id})) {
+			$b1r{$id} = $real;
+			$b1v{$id} = $virtual;
+		}
+	} else {
+		$b1r{$id} = $real;
+		$b1v{$id} = $virtual;
+}	}
 close(B1);
 
 $_ = '';
@@ -83,14 +99,23 @@ parse();
 while (<B2>) {
 	parse();
 	next unless (defined($id));
-	$b2r{$id} = $real;
-	$b2v{$id} = $virtual;
+	if(defined($b2r{$id})) {
+		print STDERR "More than one benchmark for $id in file 2\n";
+		if($real > $b2r{$id} || ($real == $b2r{$id} && $virtual > $b2v{$id})) {
+			$b2r{$id} = $real;
+			$b2v{$id} = $virtual;
+		}
+	} else {
+		$b2r{$id} = $real;
+		$b2v{$id} = $virtual;
+	}
 }
 close(B2);
 
 foreach $id (keys %b1r) {
 	if (!defined($b2r{$id})) {
 		print "Only in file 1: $id\n";
+		$onlyin1 += 1;
 		next;
 	}
 }
@@ -101,6 +126,12 @@ $n = 0;
 foreach $id (keys %b2r) {
 	if (!defined($b1r{$id})) {
 		print "Only in file 2: $id\n";
+		$onlyin2 += 1;
+		next;
+	}
+}
+foreach $id (keys %b2r) {
+	if (!defined($b1r{$id})) {
 		next;
 	}
 	my $kr = $b2r{$id} / $b1r{$id};
@@ -114,8 +145,14 @@ foreach $id (keys %b2r) {
 	$mr *= $kr;
 	$mv *= $kv;
 	$n++;
+	if ($verbose == 1) {
+		printf "Ratio:\t%.5f real, %.5f virtual\t$id\n", $kr, $kv;
+	}
+}
+if ($onlyin1 != 0 && $onlyin2 != 0) {
+	print STDERR "Converting the two benchmark files using benchmark-unify might\n";
+	print STDERR "increase the number of benchmarks which can be compared\n";
 }
-
 print "Number of benchmarks:\t\t$n\n";
 exit unless ($n);
 
-- 
1.7.7.6


Powered by blists - more mailing lists

Your e-mail address:

Powered by Openwall GNU/*/Linux - Powered by OpenVZ