Date: Thu, 19 Apr 2012 17:34:44 -0500 From: "jfoug" <jfoug@....net> To: <john-dev@...ts.openwall.com> Subject: 'New' functionality added to JTR (finding salted passwords, without salts) I have been working on some code. First 'outside' of john, and then later properly (or hacked'ly) incorporating the logic into john. This all started from that 143 million hash dump from KoreLogic. I knew a lot of them were dirty 'salted' values. I have run into this in the past. A lot of these showed up in the forums of InsidePro, and I think many of these hashes have been scooped up from there, and put into this hash dump. But the quandary, is HOW to find passwords if the hash is salted, but you only have the hash, and not the salt. Well, for a 'FEW' hash types, we can find passwords, by brute forcing the salts. Here is how I started doing it (but did not carry on fully, due to too much overhead, mostly file IO). 1. Start with a VERY good password DB (JtR's password.lst is a very good starting point). 2. Build a script that will take each word from there, and build a set of input strings, which can be tested using MD5-raw (in the 2 cases I have done, they both use MD5). 3. Run the hashes, using this large generated file as the dictionary file. 4. Carve off the salt part from the found hashes, and re-build a proper dynamic hash, using the hash, and the 'found' salt. This can simply be re-written as a 'proper' john.pot line. Well, the 2 formats I have been working on are: PHPS/dynamic_6, which is md5(md5($p).$s) with 3 byte salt, and osCommerce (dynamic_4), md5($s.$p) with a 2 byte salt. Both of these formats have salt values from a space char, up to 0x7e (the '~' tilde char). From some initial testing, I found a lot of the PHPS (I have about 800k of them), and also there are a lot of OSCommerce (I am just starting to collect them). However, using 'external' modes to search for this is VERY expensive. For the PHPS type, there are 95^3 salts, so for EACH password you are going to try, you have to md5 that password, and write 857375 lines that are that md5 hash, with 3 characters appended to it. So, building this 'dictionary' file, and then later reading it back in on the john run, takes a HELL of a lot of time, and overhead. The osCommerce is 'better' (but still very wasteful). It requires writing 9025 lines, that are 2 bytes of salt with the password appended to it, for each password. So, I have made some changes to john. 1. I added a command line param (--regen-lost-salts=# # right now can be 1 for PHPS or 2 for osCommerce). 2. I added a new 'thin' format, OSC. I did not want to hack these changes into dynamic, for dynamic_6 and dynamic_4, so it was added into the 2 thin formats, phps_fmt and the new osc_fmt. 3. When the -regen-lost-salts=1 is used, and the type is -form=phps, then the phps format will load 'raw' 32 byte hashes, without salts, and it will add a dummy salt to each one (the exact same salt). 4. If -regen-lost=2 and -form=osc was used, the exact same logic (loading raw hashes, adding a dummy salt) is done within that format. 5. Both of these formats use dynamic (dynamic_6 or dynamic_4). 6. When john loads the file, IF -regen-lost-salt was used, then john will walk the salt array, and find the last salt. This salt record will contain ALL of these just loaded 'raw' hashes, with 'fake' salts. 7. We then allocate space for 9025 salt records, or 857375 salt records (depending upon 2 byte or 3 byte salt). 8. Then each of these salt records is loaded. ALL of the data from the salt that was loaded by john, is assigned to each of these salt records. 9. Each salt record will get the 'proper' salt. So we start with ' ' then ' !' then ' @' , etc, up to '~~' (3 byte salts would be built the same way). 10. Then we simply let john run the way it wants to. We have actually inserted an extra 9025 (or 857375) salts into the runtime, and we have pointed EVERY single hash into each of these salts, so that john will be able to compare ALL hashes for each salt tested. 11. If a hash gets cracked, then the salt FROM the cracked value is shoved into the data, BEFORE the full test is done. This is because we do not yet have the 'right' salt in our likely hash. 12. The hash gets written to the .pot file as a $dyamic_4$ or a $dyamic_6$ hash (default behavior for thin formats already). These output .pot lines will have the PROPER salt added to them. 13. Within the loader, and .pot removal logic, I also had to add logic (in the -regen-lost-salts mode), so that hashes found in the .pot file are removed. This was done because at that time (loader), we still do NOT know the correct salt. Thus if the hash matches, we have to 'ignore' the salt. This change also incorporates some fixes to the logic. These hashes contain 'bad' characters. So, I was not loading data properly within the .pot loading. So I have included these changes (which really ARE a separate issue, but were found here). 1. At .pot writing, IF the format is a dynamic format, then we call a new function which will return a possibly 'fixed' hash string. 2. If the salt contains any ':' chars, or any '$' chars, then the salt gets converted into a $HEX$. This is ONLY at .pot writing. 3. Within the dynamic 'prepare' function, we reverse this logic (and remove all $HEX$ data). 4. The only time that data (the $HEX$) is left intact, is if there are NULL bytes within the HEX. If that is the case, we leave it alone and handle that HEX decoding at salt loading time 5. This 'fix' in the prepare, gets the line in proper format, to do proper comparisons at .pot loading / .pot removal. I am not quite ready to release the patch(es) just yet. I am 90% sure that the -regen-lost-salt logic is fully working, and should NOT impact normal running of john. However, I am not 100% sure of the $HEX$ to pot, and un$HEX$ at pot load is fully and bug free working. Also, I have not documented these changes at all. But I will try to get this done, this weekend at the latest. As a note, the PHPS is very slow. It will only do a couple 'passwords' a second, so you really have to have a GOOD password list, and lots of possible candidates. The OSC is much faster (95x faster). It might be fast enough to do 'some' rules logic, on VERY good password DB's and possibly a very limited amount of -inc work. The OSC is likely slower than MSCASH2. The PHPS is much slower. Jim. Content of type "text/html" skipped
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.