Date: Sun, 13 Mar 2011 14:00:28 -0500 From: "jfoug" <jfoug@....net> To: <john-dev@...ts.openwall.com> Subject: Finishing up the MD5-stuff. Well, I have also added quite a bit recently. ---------- Added ability to use field to field (from the input file). Field and username are pretty much the same (user name may get some 'domain' logic added, and I may add a 'domain' variable). Added new functions: MD5GenBaseFunc__append_fld0, MD5GenBaseFunc__append_fld1, ... MD5GenBaseFunc__append_fld9 and MD5GenBaseFunc__append2_fld0, MD5GenBaseFunc__append2_fld1, ... MD5GenBaseFunc__append2_fld9 Also added flags MGF_FLD0, MGF_FLD1, ... MGF_FLD9 Which must be set to cause the fields to be loaded at salt time. ---------- The username was already there, but now it can be forced upper or forced lower case (though flags). Added 2 new flags MGF_USERNAME_UPCASE and MGF_USERNAME_LOCASE which provide the user name, but force case when reading in the data from the input file. ---------- Added SHA1 in a limited degree, into the syntax. There are some limitations to the usage of SHA: 1. Only non-SSE code. 2. Can only directly append results (in base-16) to the other input field. So if you sha1 what is in input 1, you can only immediately append it to input2. 3. Can directly overwrite either input field. So if doing sha1 on input2 field, you can immediately overwrite input 1 or input 2 (base-16) 4. If running in SSE2 mode, the format will auto switch proper variables into x86, perform the task, and switch them back to SSE. However, this is not optimal. It is much better to properly build the script to be in x86 mode when it is needed. If all work is done in sha1, then simply set the MGF_NOTSSE2Safe flag, and SSE mode will never be entered, ever. 5. The format can overwrite the output 1. However, it will only store the first 16 bytes of the sha1 result. There are 2 functions designed to do this, and both are built to ONLY be used as the LAST step. This allows something like sha1($p) or sha1(md5($p)) to work. These functions are MD5GenBaseFunc__SHA1_crypt_input1_to_output1_FINAL and MD5GenBaseFunc__SHA1_crypt_input2_to_output1_FINAL and as you can see, they should only be used as the 'final' crypt. It would not be valid to continue from this point forward, part of the results is lost. However, it has provided john with the first 128 bits (out of 160) from the final SHA1 hash. This is 'good enough' to compare against. 6. If either of the *_FINAL functions are used, then the flag MGF_SHA1_40_BYTE_FINISH must also be used. This will cause the format to only load hashes with 40 bytes of base-16 (I will likely add base-64 support also, in near future). 7. This is still somewhat experimental. It DOES work. However it is not optimal. But it does provide the framework for looking into how to improve it. The added functions are: MD5GenBaseFunc__SHA1_crypt_input1_append_input2_base16 MD5GenBaseFunc__SHA1_crypt_input2_append_input1_base16 MD5GenBaseFunc__SHA1_crypt_input1_overwrite_input1_base16 MD5GenBaseFunc__SHA1_crypt_input2_overwrite_input2_base16 MD5GenBaseFunc__SHA1_crypt_input1_overwrite_input2_base16 MD5GenBaseFunc__SHA1_crypt_input2_overwrite_input1_base16 MD5GenBaseFunc__SHA1_crypt_input1_to_output1_FINAL MD5GenBaseFunc__SHA1_crypt_input2_to_output1_FINAL And the new 'flag' value is: MGF_SHA1_40_BYTE_FINISH Here are examples of sha1($s.$p) and sha1($p.$s) and sha1(md5$(p)) and md5(sha1($p)) [List.Generic:md5_gen(1066)] Expression=md5_gen(1066): sha1($p.$s) Flag=MGF_NOTSSE2Safe Flag=MGF_SALTED Flag=MGF_SHA1_40_BYTE_FINISH Func=MD5GenBaseFunc__clean_input Func=MD5GenBaseFunc__append_keys Func=MD5GenBaseFunc__append_salt Func=MD5GenBaseFunc__SHA1_crypt_input1_to_output1_FINAL Test=md5_gen(1066)5a12479f0a8286a832288e1dc2ea9b2eda4e382d$sG:test1 Test=md5_gen(1066)c72b6f1caddb158831cab0b08d29243ea20fc869$xxRW:thatsworking Test=md5_gen(1066)b966eff1aac95e92818a7c59326cce297b935eff$s3xx:test3 [List.Generic:md5_gen(1067)] Expression=md5_gen(1067): sha1($s.$p) Flag=MGF_NOTSSE2Safe Flag=MGF_SALTED Flag=MGF_SHA1_40_BYTE_FINISH Func=MD5GenBaseFunc__clean_input Func=MD5GenBaseFunc__append_salt Func=MD5GenBaseFunc__append_keys Func=MD5GenBaseFunc__SHA1_crypt_input1_to_output1_FINAL Test=md5_gen(1067)f5266f29ff7f1ea6fc30085c8347fcf6a6e36e9c$sG:test1 Test=md5_gen(1067)a34af873d9047541b4d76ceae7b391f0664ca99e$xxRW:thatsworking Test=md5_gen(1067)f0058038be0e821caa3031b463aed00fbe7e3beb$s3xx:test3 # md5(sha1($p)) [List.Generic:md5_gen(1063)] Flag=MGF_KEYS_INPUT Flag=MGF_SHA1_40_BYTE_FINISH Expression=md5_gen(1063): sha1(md5($p)) Func=MD5GenBaseFunc__crypt Func=MD5GenBaseFunc__SSEtoX86_switch_output1 Func=MD5GenBaseFunc__clean_input2_kwik Func=MD5GenBaseFunc__append_from_last_output_to_input2_as_base16 Func=MD5GenBaseFunc__SHA1_crypt_input2_to_output1_FINAL Test=md5_gen(1063)81d84525eb1499d518cf3cb3efcbe1d11c4ccf25:test1 Test=md5_gen(1063)6cd62e1767b65eec58d687de6d9c08a828018254:thatsworking Test=md5_gen(1063)7d653cf00d747a9fbab213b6c2b335cfe8199ff3:test3 # md5(sha1($p)) [List.Generic:md5_gen(1062)] Flag=MGF_StartInX86Mode Flag=MGF_KEYS_INPUT Expression=md5_gen(1062): md5(sha1($p)) Func=MD5GenBaseFunc__clean_input2_kwik Func=MD5GenBaseFunc__SHA1_crypt_input1_append_input2_base16 Func=MD5GenBaseFunc__X86toSSE_switch_input2 Func=MD5GenBaseFunc__crypt_in2_to_out1 Test=md5_gen(1062)a7168f0f249e3add33da11a59e228a57:test1 Test=md5_gen(1062)067dda3ad565339fffa61ba74fab0ba3:thatsworking Test=md5_gen(1062)71a1083be5c288da7e57b8c2bd7cbc96:test3 ---------- I have started to build 'thin' formats. When doing so, I am making sure that they run at least as fast as before, and some run faster. I am copying the original format, and calling it rawMD5go_thick_fmt.c (was rawMD5go_fmt.c). Thus, if there are problems, a user can back out the changes to the thin format. These are the thin formats I have done: phpass (was done before) PHPS (was done before). pixMD5 raw-md5 PO HDAA IPB2 I will probably also do SHA1-gen. raw-sha1 uses SSE, so runs much faster than the sha1 hack I have in md5-gen. I also have not looked at any of the other SHA1 based hashes, but most of them probably are already faster than what md5-gen hacking could do (at this time). Jim.
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.