Date: Thu, 17 Feb 2011 11:41:13 -0600 From: "jfoug" <jfoug@....net> To: <john-users@...ts.openwall.com> Subject: MD5 Generic improvements I have been working offline with Solar recently, due to benchmark testing being broken for the md5-gen format. The issue was found, and simple fix addressed it in the jumbo-12 update. I had a pretty large set of semi completed code improvements to md5-gen. I have taken the time to finalize these, and to add quite a few additional things. There are performance improvements (including intrinsic support, IF the intrinsic patch is applied), and some overall performance improvements, and certain sub formats improved significantly. All of these changes can be found in the patch section of the wiki, at http://openwall.info/wiki/john/patches I have made these patches against jumbo-12. It is likely that this patch will be pulled into jumbo-13. However, it is being released now, so that people can use the improvements if they so choose. Here is a list of changes. 1. Incorporation of SSE2 intrinsic code. This is triggered by the MD5_PARA #define. For this code to 'work', the SSE intrinsic patch will need to be installed, and for most performance gains, the ICC will need to be used as the compiling environment. NOTE, this intrinsic code is 'pre-patched' in. It is not part OF the intrinsic patch, but it will not compile in UNTIL the intrinsic patch code is put in. The intrinsic patch is also found on the wiki 'patches' page. 2. Incorporation of MD5_go code (faster than OpenSSL). There is a #define in the source, which will still allow the original OpenSSL code to be used (but for me, it is slower). 3. Improvements in speed for specific sub formats (re-did the scripting steps on a couple of them). 4. Improvements overall in speed of SSE (some of the intermediate level overhead reduced) 5. Several new formats (PostOffice-md5, and a format with 2 salts). Numerous new formats scripted into the john.conf file, including pixMD5 and pixMD5-salted. 6. Addition of 8 'constants' that can be used in building sub formats. There is an example for the PostOffice-md5, in the john.conf file) 7. Prior builds of md5-gen for SSE (MMX) would not be able to handle certain formats. Now the SSE builds do handle these formats (but drop back to MD5_go, x86 code). Formats which at any point within their processing build a string longer than 54 bytes, will not be usable within SSE code, since the SSE build only does a single 64 byte MD5 crypt. So a format like md5(md5($s)md5($p)) would not be valid within SSE builds. However, with the new change they do 'work', just at x86 speeds. 8. Documentation updated 9. john.conf updated. 10. Added ability to do 2-salt formats (not sure if any 2-salt formats are in actual use at this time). This 'would' allow a format to be built to do a salted format which also uses the user_id as a 2nd salt. The input file would have to be 'hand built' to do this, but it does put this type of format into the grasp of john. 11. Now, if a benchmark test fails in gen-md5, the testing continues and stays 'in' the md5-gen, testing any sub formats still needing tested. 12. Now, after the 'built in' formats are tested, the script formats in john.conf are also tested (thus, the need to get the formats updated to the newer syntax). Jim.
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.