Date: Mon, 23 May 2011 08:29:31 -0500 From: "JFoug" <jfoug@....net> To: <john-dev@...ts.openwall.com> Subject: Ideas for jumbo-6 There were a couple of ideas, which magnum and I kicked around a bit, and wrote some POC code, to perform testing. I would like to start some dialog about these, and add some other things as wish list items for further jumbo work. The things I can see in Jumbo-6 are: 1. thin formats. These will likely get released as singleton patches early on, until they get some testing. 2. Incorporate the test suite into the build process (including downloading/installing/maintaining of the test suite) 3. Change build process, so that the declaration, and init_one_format call of most formats (most in jumbo), are done automatically, thus john.c patch is NOT needed for jumbo formats. 4. Change makefile / build process, so that new formats are automatically added to the makefile OBJS= statement, and are not needed to be patched for each new format. 5. Add a new switch '-v' that will output john version information AND what john was built with. So it will show something like "John the Ripper password cracker version 1.7.7-jumbo-6\ncopywrite ....\nhomepage ...\nBuilt using [linux-x86-64]" Or something similar to this. 6. Incorporate the -v and 'Built using [XXXX]' into the test suite, so it shows what build was tested. Ok, now I would like to share some of the POC work and methods which have been tried implenting #2/#3/#4/#5 ******************* for #2, we had 4 make build targets. A test_get: a test_get_check: a test: and a test_full: test_get would use wget (other options are also being looked at, since wget may not be installed on non linux boxes), to download and install the test suite, or update it. If it could not get the data, an explanation screen would tell user how to manally get it. test_get_check: would chek to see if the test suite was installed, and if not installed, it would perform the test_get functionality. test: would first call test_get_check, to see if it needed obtained, and then run "../test/tstall quick" The quick skips some of the slower hashes. test_full: uses test_get_check to make sure the suite is there, then does "../test/tstall" This will test all formats, even the slower ones. The ./test/tstall at the top of it's script, does a cd ../test (to get it running in the proper directory, even though makefile runs in ./src). NOTE, we make install the tstall / tstone (and .bat files), into ./run, and not have the 'auto' download read them in, but have them loaded within a johh 'code' release (jumbo). This for security reasons of only releasing data files in patches. We have had a good working POC with this. Having ability to perform make & make test from within source is very nice ******************* for #3 Within make, add a rule for john.o: In this 'rule' a couple of script lines are written, using grep/sed/awk. These lines will generate 2 header files (similar to how the arch.h gets copied with proper data for the build). These 2 header files are fmt_externs.h, and fmt_registers.h There has been some discussion about how best to get the 'right' information into these headers. Several issues. For the fmt_registers.h, there are some formats which order IS wanted. We would like to keep the 'core' formats init() first, which causes them to test early on. Also, md5_gen HAS to be init() called, prior to any format using it in a thin manner. We probably want to keep dummy last, and crypt/skey as also last. However, the other order does not matter. The 2 ways to do this, are 1, 'mark' the format line within each core format with the same comment line, then grep -v remove those from inclusion into the header files. 2. rename the files for the formats which should be include. A suggestion is *_plugfmt.c for plugin formats, and *_fmt.c for the core formats. Which ever method is chosen, the core formats declares would be in john.c and then a #include fmt_externs.h and the core formats init_one() calls would be in john.c with #include "fmt_registers.h" in the middle of them. All other extern stuff or init stuff would be removed. Then the wiki 'how to build a format', would need changed to remove the information about changing the john.c (and options.c and Makefile). Also within these instructions, would list that naming of *_plugfmt.c is required (if we go that directrion). ******************* for #4 This goes along with #3. The POC designed, only worked with the *_plugfmt.c (thus, that is likely going to be a factor in choosing for #3). This is very simple. There is a build rule done, that finds all of the wildcard *_plugfmt.c files. Then a single entry at the end of the OBJS= and they are all there. Then any formats which are found be this rule are removed from the OBJS= line. From that point on, users will NOT have to worry about patching Makefile when adding a new format, it simply is there. SIDE note for #3/#4. It is very easy to pull a format OUT of john. Simply rename *_plugfmt.c to something else (such as *_plug_xx_fmt.c). Then rebuild. That format will not be included, and there should be no errors, or missed code. ******************* for #5 Has not been POC proven yet, but should be trivial. Likely each build target will have to echo out a line to a header file. The header file 'could' be arch.h. So within each target, we copy the proper arch.h, then echo "build env" >> arch.h. Then within the -v hander, we output this data. Also could show up on the 'normal' john with no command line switches screen. ******************* COMMENTS wanted. Jim PS, magnum if I missed anything, please fill in the holes. I am running on 11 hours of sleep since Thursday morning, so things are just a touch fuzzy :)
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.