john-users - Re: Couple of questions regarding rules (preprocessor) and truncation

Openwall GNU/*/Linux 3.0 - a small security-enhanced Linux distro for servers
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sun, 20 Mar 2011 04:43:58 +0300
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: Couple of questions regarding rules (preprocessor) and truncation

On Fri, Mar 18, 2011 at 08:46:55PM +0100, magnum wrote:
> Here's a cut down version of a rule line I experiment with:
> 
> [List.Rules:T1]
> >[2-9A-Z] '\p[2-9A-Z] A[0z]"1234"

BTW, you can simplify this to:

>[2-9A-Z] '\0 A[0z]"1234"

(no runtime difference, just shorter to type).

> I do want this rule to go to Z for formats that supports that length. 
> But what I would prefer, and actually did try first, is this:
> 
> >[2-*] '\p[2-*] A[0z]"1234"
> 
> This would produce a lot less *rules* when running against 
> length-impaired formats like DES or LM and many other formats too. Could 
> this be made allowed? Or is it already, using some escape I did not try?

No, this is not currently supported.

> I understand the preprocessor can't know about variables but *, + and - 
> are constants and should be known at pp time, right?

This could be implemented, but so far the preprocessor has been entirely
separate from the rule engine and I like it that way.

On my to do list, I have this item: "new reject flag: unless passwords
of length N or longer are supported".  This would let you do what you
wanted by using the new rule reject flag along with the preprocessor's
backreferences.  The syntax could be something like:

->[2-9A-Z] >\0 '\0 A[0z]"1234"

Should I treat your posting as a vote for that new rule reject flag
getting implemented sooner rather than later?

> Anyway, accepting the situation and just trying to mitigate some of the 
> duplicates, I split the example line into two:
> 
> [List.Rules:T2]
> >[2-9A-Z] '\p[2-9A-Z] A0"1234"
> >[2-9A-Z] '\p[2-9A-Z] Az"1234"
[...]
> That worked fine, 12 duplicates was muted. Lowering it to 7 works too, 
> or raising it. But why are dupes not muted if I go below 7?
> 
> user@...:~$ ./john -wo:wordlist -stdout=6 -ru:T2
> 1234lo
> 1234lo
[...]
> longca
> longca
> words: 21  time: 0:00:00:00 100.00% (ETA: Fri Mar 18 19:01:26 2011) 
> w/s: 1050  current: longca

There was a bug in the code.  I've attached the fix.  Unfortunately,
this fix has performance impact (it's an extra check to perform).

With this patch applied, or with a 32-bit build of John, I am getting:

1234lo
lo1234
lon123
long12
longc1
longca
words: 6  time: 0:00:00:00 100%  w/s: 600  current: longca

> Last, a related question: can I expect DES to behave just like 
> --stdout=8 (and LM just like --stdout=7) when it comes to muting 
> duplicates? I mean, is it the very same parts of John that does it, 
> stdout or not?

Yes.

Thanks,

Alexander

--- john-1.7.6/src/rules.c.orig	2010-02-26 01:03:44 +0000
+++ john-1.7.6/src/rules.c	2011-03-20 01:33:41 +0000
@@ -905,6 +905,8 @@ char *rules_apply(char *word, char *rule
 out_OK:
 	in[rules_max_length] = 0;
 	if (last) {
+		if (length > rules_max_length)
+			length = rules_max_length;
 		if (length >= ARCH_SIZE - 1) {
 			if (*(ARCH_WORD *)in != *(ARCH_WORD *)last)
 				return in;