Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Thu, 2 Jan 2014 00:07:16 +0100
From: Frank Dittrich <frank_dittrich@...mail.com>
To: john-dev@...ts.openwall.com
Subject: Implementing --list=format-tests (help needed for --encoding=...)

A format specific list of all valid tests might be better suited for
Alexander's fuzzing scripts, since most formats now at least have a
basic valid() which checks at least the format prefix.

That's why I implemented --list=format-tests.

$ ./john --list=format-tests --format=raw-md5*
Raw-MD5	0	5a105e8b9d40e1329780d62ea2265d8a	test1
Raw-MD5	1	$dynamic_0$378e2c4a07968da2eca692320136433d	thatsworking
Raw-MD5	2	$dynamic_0$8ad8757baa8564dc136c1e07507f4a98	test3
Raw-MD5	3	d41d8cd98f00b204e9800998ecf8427e	
Raw-MD5u	0	16c47151c18ac087cd12b3a70746c790	test1
Raw-MD5u	1	ea7ab2b5c07650badab30790d0c9b63e	
Raw-MD5u	2	f0a0b9f1dea0e458cec9a284ff434d44	�
Raw-MD5u	3	d25a0b436b768777cc9a343d283dbf5a	��
Raw-MD5u	4	719917322bf12168f8c55939e4fec8de	���
Raw-MD5u	5	d41d8cd98f00b204e9800998ecf8427e	
Raw-MD5u	6	9c3abef89ff76f8acd80eae37b35f64f	test2
Raw-MD5u	7	849ee1b88b5d887bdb058180a666b450	test3
Raw-MD5u	8	8c4cb7e8b33b56a833cdaa8673f3b425	test4
Raw-MD5u	9	537e738b1ac5551f65106368dc301ece	thatsworking

$ ./john --encoding=utf-8 --list=format-tests --format=raw-md5u
Raw-MD5u	0	16c47151c18ac087cd12b3a70746c790	test1
Raw-MD5u	1	94a4e171de16580742c4d141e6607bf7	€
Raw-MD5u	2	03c60810f0e54d16e826aca385d776c8	€€
Raw-MD5u	3	2d554433d7cde7ec8d16aaf126c3be6b	۟
Raw-MD5u	4	8007d9070b27db7b30433df2cd10abc1	ü€
Raw-MD5u	5	d41d8cd98f00b204e9800998ecf8427e	
Raw-MD5u	6	9c3abef89ff76f8acd80eae37b35f64f	test2
Raw-MD5u	7	849ee1b88b5d887bdb058180a666b450	test3
Raw-MD5u	8	8c4cb7e8b33b56a833cdaa8673f3b425	test4
Raw-MD5u	9	537e738b1ac5551f65106368dc301ece	thatsworking


As you can see, I even want to get those tests that are modified by the
format when the encoding is changed.

That's why, I called fmt_init(format) before processing the format's
tests[], and fmt_done(format) afterwards.
Without calling fmt_init and fmt_done, only dynamic formats report
different tests, depending on the encoding.

But with fmt_init and fmt_done, I get a problem whenever I want to see
the tests for --format=phpass (or any format specification that includes
phpass):

$ ./john --list=format-tests --form=phpass
Error, trying to link to $dynamic_17$ using
ciphertext=$dynamic_17$jgypwqm.JsMssPLiS8YQ00$9aaaaaSXB FAILED
$ echo $?
107

I have no idea how to avoid this problem, except by not calling
fmt_init() and fmt_done(), even if that means only dynamic formats will
report tests that depend on --encoding.

Even the git grep output doesn't give me a clue:
(bleeding-jumbo)src $ git grep -C 2 -n "Error, trying to link to"
dynamic_fmt.c-7460-     valid = pFmtLocal->methods.valid(ciphertext,
pFmtLocal);
dynamic_fmt.c-7461-     if (!valid)
dynamic_fmt.c:7462:             exit(fprintf(stderr, "Error, trying to
link to %s using ciphertext=%s FAILED
dynamic_fmt.c-7463-
dynamic_fmt.c-7464-     pFmt->params.max_keys_per_crypt =
pFmtLocal->params.max_keys_per_crypt;


So, for now, I commented out fmt_init() and fmt_done() calls.
I'll attach my patch (output of git diff), just in case anybody wants to
look into this.

Frank

diff --git a/src/listconf.c b/src/listconf.c
index 2ed40a2..a433118 100644
--- a/src/listconf.c
+++ b/src/listconf.c
@@ -70,8 +70,7 @@ static void listconf_list_options()
 	puts("formats, format-details, format-all-details, format-methods[:WHICH],");
 	// With "opencl-devices, cuda-devices, <conf section name>" added,
 	// the resulting line will get too long
-	// printf("sections, parameters:SECTION, list-data:SECTION, ");
-	puts("sections, parameters:SECTION, list-data:SECTION,");
+	puts("format-tests, sections, parameters:SECTION, list-data:SECTION,");
 #ifdef HAVE_OPENCL
 	printf("opencl-devices, ");
 #endif
@@ -578,6 +577,53 @@ void listconf_parse_late(void)
 		} while ((format = format->next));
 		exit(0);
 	}
+	if (!strncasecmp(options.listconf, "format-tests", 12)) {
+		struct fmt_main *format;
+		format = fmt_list;
+		do {
+			int ntests = 0;
+
+			/*
+			 * fmt_init() and fmt_done() required for --encoding=
+			 * support, because some formats (like Raw-MD5u)
+			 * change their tests[] depending on the encoding.
+			 *
+			 * Without fmt_init() and fmt_done(), only dynamic
+			 * formats will report different tests for
+			 * --encoding=ansi and --encoding=utf-8.
+			 *
+			 * But with fmt_init() and fmt_done(), I get an error
+			 * for --list=format-tests --format=phpass (or anything
+			 * that includes phpass, e.g. --form=cpu or --form=ph*):
+			 *
+			 * Error, trying to link to $dynamic_17$ using 
+			 * ciphertext=$dynamic_17$jgypwqm.JsMssPLiS8YQ00$9aaaaaSXB
+			 * FAILED
+			 *
+			 */
+			// fmt_init(format); // FIXME
+
+			if(format->params.tests) {
+				while (format->params.tests[ntests].ciphertext) {
+					/*
+					 * This should produce useful output for most
+					 * formats.
+					 * scrypt is the only exception, due to one
+					 * test with tabs and new lines in ciphertext
+					 * and password.
+					 */
+					printf("%s\t%d\t%s\t%s\n",
+					       format->params.label, ntests,
+					       format->params.tests[ntests].ciphertext,
+					       format->params.tests[ntests].plaintext);
+					ntests++;
+				}
+			}
+			// fmt_done(format); // FIXME
+
+		} while ((format = format->next));
+		exit(0);
+	}
 	/*
 	 * Other --list=help:WHAT are processed in listconf_parse_early(), but
 	 * these require a valid config:

Powered by blists - more mailing lists

Your e-mail address:

Powered by Openwall GNU/*/Linux - Powered by OpenVZ