Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Mon, 23 May 2011 08:29:31 -0500
From: "JFoug" <>
To: <>
Subject: Ideas for jumbo-6

There were a couple of ideas, which magnum and I kicked around a bit, and 
wrote some POC code, to perform testing.  I would like to start some dialog 
about these, and add some other things as wish list items for further jumbo 

The things I can see in Jumbo-6 are:

1. thin formats.  These will likely get released as singleton patches early 
on, until they get some testing.
2. Incorporate the test suite into the build process (including 
downloading/installing/maintaining of the test suite)
3. Change build process, so that the declaration, and init_one_format call 
of most formats (most in jumbo), are done automatically, thus john.c patch 
is NOT needed for jumbo formats.
4. Change makefile / build process, so that new formats are automatically 
added to the makefile OBJS= statement, and are not needed to be patched for 
each new format.
5. Add a new switch '-v' that will output john version information AND what 
john was built with.  So it will show something like "John the Ripper 
password cracker version 1.7.7-jumbo-6\ncopywrite ....\nhomepage ...\nBuilt 
using [linux-x86-64]"   Or something similar to this.
6. Incorporate the -v and 'Built using [XXXX]' into the test suite, so it 
shows what build was tested.

Ok, now I would like to share some of the POC work and methods which have 
been tried implenting #2/#3/#4/#5

for #2, we had 4 make build targets.  A test_get: a test_get_check: a test: 
and a test_full:

test_get would use wget (other options are also being looked at, since wget 
may not be installed on non linux boxes), to download and install the test 
suite, or update it.  If it could not get the data, an explanation screen 
would tell user how to manally get it.

test_get_check: would chek to see if the test suite was installed, and if 
not installed, it would perform the test_get functionality.

test: would first call test_get_check, to see if it needed obtained, and 
then run "../test/tstall quick"   The quick skips some of the slower hashes.

test_full: uses test_get_check to make sure the suite is there, then does 
"../test/tstall"  This will test all formats, even the slower ones.

The ./test/tstall  at the top of it's script, does a cd ../test  (to get it 
running in the proper directory, even though makefile runs in ./src).  NOTE, 
we make install the tstall / tstone (and .bat files), into ./run, and not 
have the 'auto' download read them in, but have them loaded within a johh 
'code' release (jumbo).   This for security reasons of only releasing data 
files in patches.

We have had a good working POC with this. Having ability to perform make & 
make test  from within source is very nice

for #3

Within make, add a rule for john.o:  In this 'rule' a couple of script lines 
are written, using grep/sed/awk.  These lines will generate 2 header files 
(similar to how the arch.h gets copied with proper data for the build). 
These 2 header files are fmt_externs.h, and fmt_registers.h

There has been some discussion about how best to get the 'right' information 
into these headers.  Several issues.  For the fmt_registers.h, there are 
some formats which order IS wanted.  We would like to keep the 'core' 
formats init() first, which causes them to test early on.  Also, md5_gen HAS 
to be init() called, prior to any format using it in a thin manner.  We 
probably want to keep dummy last, and crypt/skey as also last.  However, the 
other order does not matter.   The 2 ways to do this, are 1, 'mark' the 
format line within each core format with the same comment line, then grep -v 
remove those from inclusion into the header files.  2. rename the files for 
the formats which should be include.  A suggestion is *_plugfmt.c for plugin 
formats, and *_fmt.c for the core formats.

Which ever method is chosen, the core formats declares would be in john.c 
and then a #include fmt_externs.h  and the core formats init_one() calls 
would be in john.c with #include "fmt_registers.h" in the middle of them. 
All other extern stuff or init stuff would be removed.

Then the wiki 'how to build a format', would need changed to remove the 
information about changing the john.c  (and options.c and Makefile).   Also 
within these instructions, would list that naming of *_plugfmt.c is required 
(if we go that directrion).

for #4

This goes along with #3.  The POC designed, only worked with the *_plugfmt.c 
(thus, that is likely going to be a factor in choosing for #3).   This is 
very simple.  There is a build rule done, that finds all of the wildcard 
*_plugfmt.c files.  Then a single entry at the end of the OBJS= and they are 
all there. Then any formats which are found be this rule are removed from 
the OBJS= line.  From that point on, users will NOT have to worry about 
patching Makefile when adding a new format, it simply is there.

SIDE note for #3/#4.  It is very easy to pull a format OUT of john.  Simply 
rename *_plugfmt.c to something else (such as *_plug_xx_fmt.c).  Then 
rebuild.  That format will not be included, and there should be no errors, 
or missed code.

for #5

Has not been POC proven yet, but should be trivial.  Likely each build 
target will have to echo out a line to a header file. The header file 
'could' be arch.h.  So within each target, we copy the proper arch.h, then 
echo "build env" >> arch.h.    Then within the -v hander, we output this 
data.  Also could show up on the 'normal' john with no command line switches 


COMMENTS wanted.


PS, magnum if I missed anything, please fill in the holes.  I am running on 
11 hours of sleep since Thursday morning, so things are  just a touch fuzzy 

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.