Date: Tue, 27 Oct 2009 14:31:19 -0500 From: "JFoug" <jfoug@....net> To: <john-users@...ts.openwall.com> Subject: My patches (Performance, options, mingw, vc, generic md5, phpass, etc) I have unified all of my released patches, into a single patch. This patch was derived from 220.127.116.11 after jumbo-1 patch is applied This can be found on the wiki at: http://openwall.info/wiki/john/patches (there is also an update to the md5-generic test files). It is not just all of the prior patches packed into one, but it has pretty significant improvements to the generic MD5 code. In the MD5, I have gotten things working as fast (if not faster), than the 'native' functions. Also, there is a way to link an existing format (or a new one), into the MD5 generic code, as an 'engine' to do the work. Thus, you can create a very 'thin' format (only having 4 functions and a conversion function) to build, and those functions are pretty simple (validate, init (which now actually DOES something), getsalt and getbinary). I have coded phpass and PHPS formats to use the generics. There are many others which 'could' be done, but that is where I started. There is now a way to build new formats into generic md5, WITHOUT having to add any code into john. There is no parser, but lines can be added to john.conf that properly tells the md5 generic engine which 'primitive' functions to call, and provide 'testing' data. The entire process is documented within the updated john.conf file (along with a handful of sample 'formats'). There is also a doc/MD5_GENERIC readme file. All formats under 1000 are reserved as john built-in's. 1000 and above are free to use (by adding them to john.conf). There are many new command line options (as were in one of my prior patches). However, there is another new option. That option is --field-separator-char=c | \xHH By default, john uses a ':' char (the colon). This char is used in candidate files, in the john.pot, etc. With this new option, you can specify (on the command line), to use a different character than the colon. Thus, for formats which have a salt that uses the : char (or userID's with the : embedded), you can create an input file, replacing all of the 'separator' characters with a new character, and then on the command line, specify to john that this new character is the separator character to use. NOTE this character also ends up in the john.pot file, AND lines in the john.pot file that do NOT use this separator (say some lines with a : character) will NOT be read as valid john.pot lines (when restoring). There is one format, md5_gen(7) which requires usage of the --field-separator-char= switch. That format is specfically for vBulletin format, with 3 byte salt, and all chars from space to ~ being valid to be in the salt (thus the : character is a valid salt char). In the 'test' file gen_7_tst.in (which is md5_gen(7) format), the file requires ASCII 31, or --field-separator-char=\x19 to be used, or nothing is read from the file. If there are any patch issues, compile issues (I know there are warnings building under OSX, but I do not get the warnings, so somone please list them), or of there are run time issues, then please follow up. One BIG thing I saw in testing, was that the generic md5 (and the 'fast' MD5) are NOT working for MMX. They work for generic x86 and for SSE2 (tested only with MinGW, and with VC for x86 only, as that is all I have). So you you normally build with MMX, this patch should NOT be installed at this time. I did have MMX (and SSE2) working at one time, so I will have to dig through changes, and see where it broke at, and why. I have not done testing on using MMX, since it is really not that much improvement over generic, AND SSE2 is close to twice as fast as the MMX (when I had MMX working) Jim.
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.