john-users - john improvement suggestions

Openwall wordlists collection for password cracking (20+ languages)
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Mon, 19 Dec 2005 21:48:30 +0000 (UTC)
From:  Radim Horak <yesbody@...nam.cz>
To: john-users@...ts.openwall.com
Subject:  john improvement suggestions

Hi everyone
and very special thanks to Solar Designer for the great John :)
I've discovered this forum only recently, but I've been using John for quite 
some time now, so I would like to share some ideas:

1. Bugs and annoyances
- I have passwords (Traditional DES) from some old linux box, that are longer 
than 16 chars, ie. consist of 3 hashes (crypt24?). John ignores such passwords 
completely. I have tested them by manually cutting them. The 3rd hash uses salt 
from the beginning of 2nd hash as 2nd hash uses the salt from beginning of the 
1st hash. I cannot provide the hashes nor I have access to that old linux box.
- Is it possible to disable log to avoid very large log files (with certain 
rules)?

2. Compilation
- I have tested MSVC compiler with john.  
(freely avail. here: http://msdn.microsoft.com/visualc/vctoolkit2003/)
I compiled the encryption routines with these parameters:
cl /c /TC /G7 /Ox /O2 LM_fmt.c
and the rest (I guess more POSIX dependent files) under the cygwin. The result 
was notable improvement in Lanman DES cracking speed (around 19% increase), on 
AthlonXP cpu (comparing to gcc -O3 -march=athlon-xp). Other algorithms were 
performing at the same speed.
- At this point, I'd like to see some way to increase the benchmarking time to 
get more coherent/stable/exact figures.

3. Optimizations
- In bigcrypt and other splitted psw. hashes, the first part(s) of the password 
has known length of 8 (7 in LM). When cracking such passwords, the password 
candidates (either from dictionary or incremental mode) that are shorter should 
be filtered out (skipped).
- Optimize regexp expansion order [a-z] according to some general/usage 
statistics.
- When there are psw. hashes cracked in session, does john skip cracking them 
immediatelly? Or is there a session reload needed?

4. Other improvements
- I use adhoc "wordlist" rules a lot, could there be some kind of this 
functionality: -rules:wordlist_set_1 for selecting wordlist rules??
- Is it possible to execute mmx and non-mmx instructions in x86 CPUs in 
parallel? If so would it be possible get advantage of it in john?
- The cygwin version of john can't access files (ie. wordlists) larger then 2 
GB, would you consider a more win32 native/friendly version of john? (mingw?)
- I think that the complexity gap between wordlist mode (even with rules) and 
incremental modes is too big, there is a "undiscovered country" of various 
optimizations/complexity reductions etc. Ie. like generating only "pronouncable" 
passwords (http://www.multicians.org/thvv/gpw.html) or the recent addition of 
alpha-numeric charset. Another good thing to do would be to re-feed the cracked 
passwords with the single-mode rules.

5. Time-memory trade-off for Traditional DES
- I know. I know that there are salts that make it less effective, but I think 
it's still worth (I assume 4096-times more memory needed for the equivalent of 
non-salt performance, or 1024-times more memory for 1/16th of non-salt 
performance. 
Ophcrack (http://lasecwww.epfl.ch/~oechslin/projects/ophcrack/) uses 701 MB of 
tables for alphanumeric LM passwords which is comparable to smallalphanumeric 
Traditional Unix DES and 701 GB is quite acceptable storage space today, much 
more tomorrow. Even pure smallalphabetical table would be beneficial and much 
less demanding.)

6. There was mentioned a non-public sse-optimized version of john in this forum. 
 Is there a chance we (public) will ever get our hands on it? If so, when?

I hope to inspire
and thanks in advance for any answers
-Radim