Date: Mon, 25 Aug 2008 11:01:11 +0400 From: Solar Designer <solar@...nwall.com> To: john-users@...ts.openwall.com Subject: jumbo patch for 22.214.171.124 Hi, I've just released the jumbo patch updated to JtR 126.96.36.199, with newly contributed modules added and lots of bugs fixed (both in the newly added code and in code previously found in jumbo patches), and also with some generic improvements that I originally introduced in JtR Pro. The current revision is 188.8.131.52-all-2, linked right from JtR homepage and also found in the contrib/ FTP directory: http://www.openwall.com/john/ ftp://ftp.openwall.com/pub/projects/john/contrib/ It has been briefly tested on all four combinations of 32-bit / 64-bit and little-endian / big-endian, as well as with / without generic / MMX / SSE2 assembly code on x86 and x86-64. Some more detail on the changes, besides the update to JtR 184.108.40.206: I have revised Alain's NTLM x86-64 assembly code to use instruction pointer relative addressing, which made it work on Mac OS X (originally in JtR Pro). Thus, the extra macosx-* make targets introduced in 1.7.3+ are now also usable with the jumbo patch (tested on 10.5). However, I did not revise Alain's and others' x86 and x86-64 assembly codes for compilation with Sun's assembler. Thus, the extra solaris-x86-* make targets introduced in 1.7.3+ are not supposed to work when the jumbo patch is applied, at least not yet. The NTLM "loader hack" previously found in jumbo patches has been replaced with more correct code from JtR Pro. However, it is surrounded by pieces of code derived from the older "loader hack" for the various NET*LM (challenge/response) "formats". The Mac OS X 10.4+ salted SHA-1 hashes support code has been replaced with my own, from JtR Pro. It is not very fast yet (it uses OpenSSL), but it is cleaner. The corresponding "format label" is now "XSHA" (or "xsha" if you like), and only all-uppercase strings will be accepted for the hashes (48 hex digits), because that's how Mac OS X itself encodes those hashes. The OpenSSL-derived implementation of SHA-1 has been dropped, in favor of using one from whatever version of OpenSSL we're linking against. We already had a dependency on OpenSSL in the jumbo patches anyway. Several "formats" have been revised to use OpenSSL's newer names and calling conventions for the DES routines. The code still builds and works with OpenSSL 0.9.7, but now it should also build and work with later/future versions, and with some distribution vendor adjusted ones that dropped the backwards compatibility stuff earlier (e.g., NetBSD). Many contributed additions and fixes have been merged: - sap friend's "patch for SAP-passwords (BCODE & PASSCODE)", john-1.7.2-SAPLover-1.diff, with minor bug fixes, and sap_prepare.pl is now added to the "run" directory; - Samuel Monux's "NetscreenOS passwords", john-1.7.2-netscreen-2.diff, with minor changes, and netscreen.py is now added to the "run" directory; - the portion of Simon Marechal's "working with many passwords", john-1.7.2-largepasswordset-1.diff, responsible for faster processing of successful guesses; - Bucsay Balazs and Peter Kasza's "raw-md5 module improvement" and revised "new MYSQL323 module", john-1.7.3-rawMD5_fast-2.diff, john-220.127.116.11-rawmd5-bigendian.patch, and john-1.7.3-MYSQL_fast-2.diff, with minor bug fixes and a symbol clash resolved; - JoMo-Kun's bug fix to his loader hack, "NetLM / NetNTLM loader.c Patch", loader.c.diff. Oh, did I mention bug fixes? ;-) Well, the most common bug, of those I dealt with this time, was to declare something as an array of char, then cast portions of it to (int *) or the like (and dereference). Indeed, this works on x86, but it does not guarantee proper alignment - so things break on some builds on architectures with strict alignment requirements, and performance is sometimes non-optimal even on x86. An even worse variation of the bug is to inconsistently cast to (int *) and (long *). This works on 32-bit, sometimes works on 64-bit little-endian, but breaks badly (or rather nicely, because it is easy to reproduce) on 64-bit big-endian. The fix I used this time was to change the initial declaration to be based on a large enough integer data type (such as ARCH_WORD_32), then cast to (char *) or the like in places where the code depended on that. Other bugs included out-of-bounds array accesses - on one occasion it was a cross-function off-by-one nicely spotted at compile time by gcc 4.3.1 (impressive), on another it was an issue with OpenSSL DES routines processing data in whole DES blocks, whereas the buffers were not a multiple of DES block size. Speaking of new code, sapB_fmt.c incorrectly declared final_key as BINARY_SIZE+1 instead of BINARY_SIZE*2 bytes. A bug that cost me half an hour to figure out was the KRB5_* stuff making an assumption about the size of DES key schedule structs, which may differ across OpenSSL versions and builds. I'm sure there are plenty of bugs left in the jumbo patch, but at least all self-tests pass on all machines and make targets that I used for testing this time. Not surprisingly, not a single bug was found in "official" JtR code during this testing session. ;-) I know there's still plenty of stuff to consider and maybe merge from Simon Marechal's "myjohn". Simon - would you be so kind to submit it as patches against 18.104.22.168-all-2 (or whatever will be latest at the time)? When submitting patches, please pay attention to these guidelines: http://openwall.info/wiki/how-to-make-patches Also being made available at the same time is "revision 1" of the patch for 22.214.171.124, which is a mere update to this newer version of JtR - without any additions or fixes. It is only available via FTP, under contrib/historical/ - because it is not meant for actual use, but rather to maintain some "poor man's revision history". (Maybe we'll introduce proper revision control for contributed patches eventually, in one way or another. Meanwhile, keeping older and intermediate revisions under historical/ should do. Right now, only "official" JtR code is in our CVS repository, along with other Openwall software.) And last but not least - Thank You to all who have contributed code or provided bug reports or fixes. Alexander -- To unsubscribe, e-mail john-users-unsubscribe@...ts.openwall.com and reply to the automated confirmation request that will be sent to you.
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.