[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sun, 3 Jan 2010 23:36:27 +0300
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: JtR 1.7.4 and jumbo patch update
Regarding Matt's benchmark (of 1.7.3.4's rules engine vs. 1.7.4's),
which revealed a bug in 1.7.4:
On Sun, Dec 27, 2009 at 08:01:42PM -0500, Charles Weir wrote:
> A copy of the config file can be obtained from the following link:
>
> http://sites.google.com/site/reusablesec/Home/john-the-ripper-files/john-the-ripper-sample-configs-1
>
> For the input dictionary I used one of the lowercase English
> dictionaries available on the openwall ftp site, (I think it was the
> large one). The dictionary contained 444,678 words.
/pub/wordlists/languages/English/4-extra/lower.gz contains 444,678 lines
(a few of which are comments rather than words), so I think it was this
one. The -extra wordlists don't encompass the smaller and higher
quality ones, so they contain relatively obscure and questionable
"words" only. It is not a good idea to use one of them on its own
(other than after having run through one or more of the higher quality
wordlists, such as those found under -large). I thought this was
obvious from the naming ("extra" is just that), the comments in the
files, and the actual content, but perhaps this needs to be documented
explicitly.
Anyhow, the -extra wordlist is OK for a test run when there's no goal to
actually crack passwords.
> Running JtR version 1.7.3.4
> Ryoki:run cweir$ ./john
> -wordlist=../../../custom/dictionaries/english-lower -rules -stdout > /dev/null
[...]
> words: 10495949352 time: 0:01:04:26 100% w/s: 2714K current: 9zzzzzzzthi$
>
> ------------------------------------------------------------------
> Running JtR version 1.7.4
> Ryoki:run cweir$ ./john
> -wordlist=../../../custom/dictionaries/english-lower -rules -stdout > /dev/null
[...]
> words: 10495945056 time: 0:00:49:48 100% w/s: 3512K current: 9zzzzzzzthi$
[...]
> JtR 1.7.4 ran noticeably faster than JtR 1.7.3.4, completing its
> session in 76% of the time it took 1.7.3.4 to finish. The one anomaly
> was that the 1.7.4 session outputted that it made 10,495,945,056
> guesses, while the 1.7.3.4 session outputted that it made
> 10,495,949,352 guesses. The difference in guesses may have just been a
> reporting issue, (aka the final count might not be updated), but I'll
> leave it to someone more knowledgeable to answer that question.
As I pointed out before, this indicated that there was a bug somewhere,
and I provided a relevant patch, john-1.7.4-last-fix.diff, attached to
one of my postings. Since there were no followups, I did not know
whether the above discrepancy was caused by the bug fixed by the patch
or not. Today, I went to reproduce the issue myself, and I did. For
this, I needed the -extra wordlist mentioned above (first 10k lines of
it proved to be enough), Matt's ruleset (with all instances of [0-9]
changed to [0-1] for quicker test runs), and a 64-bit build of JtR (the
problem was not reproducible with a 32-bit build). The conclusion was
that john-1.7.4-last-fix.diff did not fix the bug (it fixed another
bug), and I came up with john-1.7.4-last-fix-bis.diff (attached to this
message) that fixes the problem identified in Matt's test runs above (at
least the way I reproduced it).
I've just released 1.7.4-jumbo-5 adding the extra hunk from
-last-fix-bis mentioned above, and indeed I am going to include a fix in
the next version of JtR.
BTW, I am not relying on the "word count" alone in my testing. Rather
than redirect the output of JtR to /dev/null, I pipe it into md5sum, and
I make sure there are no unexpected changes in the MD5 digest of JtR's
output between versions.
Thanks,
Alexander
diff -urp john-1.7.4/src/rules.c john-1.7.4-mod/src/rules.c
--- john-1.7.4/src/rules.c 2009-12-24 05:02:52 +0000
+++ john-1.7.4-mod/src/rules.c 2010-01-03 18:26:56 +0000
@@ -258,7 +258,7 @@ void rules_init(int max_length)
rules_init_length(max_length);
}
-char *rules_reject(char *rule, int split, struct db_main *db)
+char *rules_reject(char *rule, int split, char *last, struct db_main *db)
{
static char out_rule[RULE_BUFFER_SIZE];
@@ -309,7 +309,7 @@ char *rules_reject(char *rule, int split
accept:
rules_pass--;
strnzcpy(out_rule, rule - 1, sizeof(out_rule));
- rules_apply("", out_rule, split, NULL);
+ rules_apply("", out_rule, split, last);
rules_pass++;
return out_rule;
@@ -905,15 +905,13 @@ out_OK:
if (length >= ARCH_SIZE - 1) {
if (*(ARCH_WORD *)in != *(ARCH_WORD *)last)
return in;
- if (!strcmp(&in[ARCH_SIZE - 1], &last[ARCH_SIZE - 1]))
- goto out_NULL;
- return in;
+ if (strcmp(&in[ARCH_SIZE - 1], &last[ARCH_SIZE - 1]))
+ return in;
+ return NULL;
}
- if (last[2])
- return in;
- if (in[0] != last[0])
+ if (last[length])
return in;
- if (in[1] != last[1] && length)
+ if (memcmp(in, last, length))
return in;
return NULL;
}
@@ -967,7 +965,7 @@ int rules_check(struct rpp_context *star
rules_pass = -1; /* rules_reject() will turn this into -2 */
while ((rule = rpp_next(&ctx))) {
- rules_reject(rule, split, NULL);
+ rules_reject(rule, split, NULL, NULL);
if (rules_errno) break;
if (ctx.input) rules_line = ctx.input->number;
diff -urp john-1.7.4/src/rules.h john-1.7.4-mod/src/rules.h
--- john-1.7.4/src/rules.h 2009-12-24 02:38:12 +0000
+++ john-1.7.4-mod/src/rules.h 2009-12-28 19:28:21 +0000
@@ -54,8 +54,11 @@ extern void rules_init(int max_length);
*
* split == 0 "single crack" mode rules allowed
* split < 0 "single crack" mode rules are invalid
+ *
+ * last may specify which internal buffer must not be touched.
*/
-extern char *rules_reject(char *rule, int split, struct db_main *db);
+extern char *rules_reject(char *rule, int split, char *last,
+ struct db_main *db);
/*
* Applies rule to a word. Returns the updated word, or NULL if rejected or
diff -urp john-1.7.4/src/single.c john-1.7.4-mod/src/single.c
--- john-1.7.4/src/single.c 2009-12-24 00:41:54 +0000
+++ john-1.7.4-mod/src/single.c 2009-12-28 19:27:22 +0000
@@ -309,7 +309,7 @@ static void single_run(void)
saved_min = rec_rule;
while ((prerule = rpp_next(rule_ctx))) {
- if (!(rule = rules_reject(prerule, 0, single_db))) {
+ if (!(rule = rules_reject(prerule, 0, NULL, single_db))) {
log_event("- Rule #%d: '%.100s' rejected",
++rule_number, prerule);
continue;
diff -urp john-1.7.4/src/wordlist.c john-1.7.4-mod/src/wordlist.c
--- john-1.7.4/src/wordlist.c 2009-12-24 00:42:27 +0000
+++ john-1.7.4-mod/src/wordlist.c 2009-12-28 19:28:07 +0000
@@ -194,7 +194,7 @@ void do_wordlist_crack(struct db_main *d
if (prerule)
do {
if (rules) {
- if ((rule = rules_reject(prerule, -1, db))) {
+ if ((rule = rules_reject(prerule, -1, last, db))) {
if (strcmp(prerule, rule))
log_event("- Rule #%d: '%.100s'"
" accepted as '%.100s'",
Powered by blists - more mailing lists
Powered by Openwall GNU/*/Linux -
Powered by OpenVZ
