john-users - Re: JtR 1.7.4 and jumbo patch update

Openwall wordlists collection for password cracking (20+ languages)
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 28 Dec 2009 23:14:51 +0300
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: JtR 1.7.4 and jumbo patch update

On Sun, Dec 27, 2009 at 08:01:42PM -0500, Charles Weir wrote:
> Here is some benchmark data for running 1.7.3.4 and 1.7.4 on MacOSX
> 10.6.2 Snow Leopard:
> 
> Goal: To evaluate the running time differences between JtR 1.7.3.4 and
> 1.7.4 due to modifications made in 1.7.4 with regard to how word
> mangling rules are implemented.

Yes, thank you!  This is precisely what I wanted - this kind of testing
and benchmarks, especially with non-default rulesets.

> http://sites.google.com/site/reusablesec/Home/john-the-ripper-files/john-the-ripper-sample-configs-1

Thank you for sharing this.  As you're aware, this ruleset might produce
lots of duplicate candidate passwords, especially with length-limited
and/or case-insensitive hashes.  I don't think adding some "-c" flags
and some "<*" and "<-" commands would make it a lot less readable, so
you could want to do that.  Also, if you have ":" (no-op), you need to
use "l Q" instead of just "l", and ditto for "c" and "u".  One thing
that is completely unclear to me is your use of "Ct" instead of just "c" -
why is that?

BTW, with 1.7.4 you can make the ruleset shorter.  For example, the
following lines:

/a lsa@
/e lse3
/l lsl1
/o lso0
/s lss$

may be replaced with:

/[aelos] l s\0\p[@...$]

> Running JtR version 1.7.3.4
[...]
> words: 10495949352  time: 0:01:04:26 100%  w/s: 2714K  current: 9zzzzzzzthi$

> Running JtR version 1.7.4
[...]
> words: 10495945056  time: 0:00:49:48 100%  w/s: 3512K  current: 9zzzzzzzthi$

> JtR 1.7.4 ran noticeably faster than JtR 1.7.3.4, completing its
> session in 76% of the time it took 1.7.3.4 to finish. The one anomaly
> was that the 1.7.4 session outputted that it made 10,495,945,056
> guesses, while the 1.7.3.4 session outputted that it made
> 10,495,949,352 guesses. The difference in guesses may have just been a
> reporting issue, (aka the final count might not be updated), but I'll
> leave it to someone more knowledgeable to answer that question.

No, it's not just a reporting issue.  More like a bug, and I have in
fact just found and fixed a relevant bug.  Can you please repeat the
test of 1.7.4 with the attached patch?

Please make no changes to the ruleset for this test yet.  We need to
make sure the bug is fixed first.

Thanks again,

Alexander

diff -urp john-1.7.4/src/rules.c john-1.7.4-last-fix/src/rules.c
--- john-1.7.4/src/rules.c	2009-12-24 05:02:52 +0000
+++ john-1.7.4-last-fix/src/rules.c	2009-12-28 19:33:28 +0000
@@ -258,7 +258,7 @@ void rules_init(int max_length)
 	rules_init_length(max_length);
 }
 
-char *rules_reject(char *rule, int split, struct db_main *db)
+char *rules_reject(char *rule, int split, char *last, struct db_main *db)
 {
 	static char out_rule[RULE_BUFFER_SIZE];
 
@@ -309,7 +309,7 @@ char *rules_reject(char *rule, int split
 accept:
 	rules_pass--;
 	strnzcpy(out_rule, rule - 1, sizeof(out_rule));
-	rules_apply("", out_rule, split, NULL);
+	rules_apply("", out_rule, split, last);
 	rules_pass++;
 
 	return out_rule;
@@ -967,7 +967,7 @@ int rules_check(struct rpp_context *star
 
 	rules_pass = -1; /* rules_reject() will turn this into -2 */
 	while ((rule = rpp_next(&ctx))) {
-		rules_reject(rule, split, NULL);
+		rules_reject(rule, split, NULL, NULL);
 		if (rules_errno) break;
 
 		if (ctx.input) rules_line = ctx.input->number;
diff -urp john-1.7.4/src/rules.h john-1.7.4-last-fix/src/rules.h
--- john-1.7.4/src/rules.h	2009-12-24 02:38:12 +0000
+++ john-1.7.4-last-fix/src/rules.h	2009-12-28 19:28:21 +0000
@@ -54,8 +54,11 @@ extern void rules_init(int max_length);
  *
  * split == 0	"single crack" mode rules allowed
  * split < 0	"single crack" mode rules are invalid
+ *
+ * last may specify which internal buffer must not be touched.
  */
-extern char *rules_reject(char *rule, int split, struct db_main *db);
+extern char *rules_reject(char *rule, int split, char *last,
+	struct db_main *db);
 
 /*
  * Applies rule to a word. Returns the updated word, or NULL if rejected or
diff -urp john-1.7.4/src/single.c john-1.7.4-last-fix/src/single.c
--- john-1.7.4/src/single.c	2009-12-24 00:41:54 +0000
+++ john-1.7.4-last-fix/src/single.c	2009-12-28 19:27:22 +0000
@@ -309,7 +309,7 @@ static void single_run(void)
 
 	saved_min = rec_rule;
 	while ((prerule = rpp_next(rule_ctx))) {
-		if (!(rule = rules_reject(prerule, 0, single_db))) {
+		if (!(rule = rules_reject(prerule, 0, NULL, single_db))) {
 			log_event("- Rule #%d: '%.100s' rejected",
 				++rule_number, prerule);
 			continue;
diff -urp john-1.7.4/src/wordlist.c john-1.7.4-last-fix/src/wordlist.c
--- john-1.7.4/src/wordlist.c	2009-12-24 00:42:27 +0000
+++ john-1.7.4-last-fix/src/wordlist.c	2009-12-28 19:28:07 +0000
@@ -194,7 +194,7 @@ void do_wordlist_crack(struct db_main *d
 	if (prerule)
 	do {
 		if (rules) {
-			if ((rule = rules_reject(prerule, -1, db))) {
+			if ((rule = rules_reject(prerule, -1, last, db))) {
 				if (strcmp(prerule, rule))
 					log_event("- Rule #%d: '%.100s'"
 						" accepted as '%.100s'",