Date: Tue, 25 Jan 2011 10:46:27 -0500 From: rs904c@...scape.net To: john-users@...ts.openwall.com Subject: Re: Password file hash type analysis in JtR John Users/Frank/Alex, I was thinking of this as being a pre-analysis type tool, not post cracking. So, before any cracking begins, I would want to know which hash formats I have in the file. This way I will make sure and manually run JtR on each of the other hash formats, if any, and not only on the first hash format JtR finds in the file. I think the JtR executable with a command-line parameter (like: --analysis ) could do this more efficiently and cleanly, than a “For Loop”. JtR is already doing this for the first hash format it finds, then it starts cracking. I'd like to see it review all the hash formats, and just report which ones are contained in the file (with optional line numbers) and any it does not recognize. Does this seem useful to you all, or just me? -Robert B. Harris from VA -----Original Message----- From: Frank Dittrich <frank_dittrich@...mail.com> To: john-users@...ts.openwall.com Sent: Tue, Jan 25, 2011 4:56 am Subject: Re: [john-users] Password file hash type analysis in JtR Robert Harris wrote: I know that Unix/Linux and Windows password files can have multiple hash types in them, I think the following output/utility would be useful in tR. What is your use case? >>> A Unix system file with multiple hash types used and supported. This could also have been used for the contest held this summer. Potential new output: The following password hash format(s) was/were discovered in the input ile <file>: <format1> <format2> ... <formatx> Is that just one line describing the format? s you would specify it on the command line (with --format=) or as john ould report it (DES vs. Traditional DES, BF vs. OpenBSD Blowfish, ...)? >>> Both would be nice. > Line(s) 1-30 contained <format1> type passwords Line(s) 31-60 contained <format2> type passwords ... Line(s) 61-90 contained <formatx> type passwords Without listing the corresponding hashes/passwords? >>> Yes, I really wouldn't need the hash information, the line numbers would probably due fine. Depending on your needs, something like this might work: sample pot file: $LM$44efce164ab921ca:123456 0xCP/g3c1sQc:password 0ewVEw7R2GAw:password 0vrzSg43HfWI:123456 0xCP/g3c1sQc:password 0.6.TrD/0sas:trustno1 rEJ1uz6ul/WU:trustno1 1$12$1YZMQPfPxS3.kStGKUBCO.:1111 10adc3949ba59abbe56e057f20f883e:123456 LM$b757bf5c0d87772f:1234 cut -d: -f 1 -s john.pot|grep -n "^" > cracked_hashes cracked_hashes would then look like this: 1:$LM$44efce164ab921ca :10xCP/g3c1sQc :20ewVEw7R2GAw :10vrzSg43HfWI :10xCP/g3c1sQc :10.6.TrD/0sas :trEJ1uz6ul/WU :$1$12$1YZMQPfPxS3.kStGKUBCO. :e10adc3949ba59abbe56e057f20f883e 0:$LM$b757bf5c0d87772f ename your john.pot file before continuing. Actually, the jumbo-patched john has an option --pot=NAME, ut I never used it... That's why I just renamed john.pot) The next step probably requires the jumbo patch, due to --show=LEFT. And, for our use case, it requires an empty pot file.) for f in DES BSDI MD5 BF AFS LM; do ./john --format=$f --show=LEFT racked_hashes > cracked_hashes_$f; done You may also write it across multiple lines, the shell will prompt you or the continuation: for f in DES BSDI MD5 BF AFS LM do ./john --format=$f --show=LEFT cracked_hashes > cracked_hashes_$f done Check how the hashes are distributed according to hash type: wc -l cracked_hashes_* wc -l cracked_hashes_* 0 cracked_hashes_AFS 0 cracked_hashes_BF 0 cracked_hashes_BSDI 5 cracked_hashes_DES 4 cracked_hashes_LM 1 cracked_hashes_MD5 10 total On first glance, this looks good. However, I know I included 6 DES hashes (one duplicate). :10xCP/g3c1sQc :10xCP/g3c1sQc Just 5 of these appear in cracked_hashes_DES, because the duplicate as been removed. I'm not sure if this is unintended or not. f a user wants to remove cracked hashes, but nevertheless ontinue to use --single mode, keeping the duplicates might be better.) o, where does the additional hash come from? I included 2 LM hashes: :$LM$44efce164ab921ca 0:$LM$b757bf5c0d87772f cracked_hashes_LM, however, contains (in addition to the expected content): :2:$LM$be56e057f20f883e :1:$LM$e10adc3949ba59ab These 2 additional lines are the first and second half of the hash 10adc3949ba59abbe56e057f20f883e in line 9 of my original racked_hashes file. While john stores LM hashes with a preceding $LM$ marker, he algorithm internally accepts output generated by other tools. hat's why there's an ambiguity.) e10adc3949ba59abbe56e057f20f883e is a raw-MD5 hash which didn't ppear in any cracked_hashes_* file, because I just used the format list f the unpatched john. Using the complete list: for f in DES BSDI MD5 BF AFS LM NT XSHA PO raw-MD5 MD5-gen \ IPB2 raw-sha1 md5a hmac-md5 phpass-md5 KRB5 bfegg \ nsldap ssha openssha oracle oracle11 MYSQL \ mysql-sha1 mscash lotus5 DOMINOSEC \ NETLM NETNTLM NETLMv2 NETNTLMv2 NETHALFLM \ mssql mssql05 epi phps mysql-fast pix-md5 sapG \ sapB md5ns HDAA DMD5 crypt do ./john --format=$f --show=LEFT cracked_hashes > cracked_hashes_$f done produces this output (to stderr): Generic crypt(3) module: hash encoding string length 20, type id $L ppears to be unsupported on this system; will not load such hashes. eneric crypt(3) module: hash encoding string length 32, type id #0 ppears to be unsupported on this system; will not load such hashes. And this list of files: $ wc -l cracked_hashes_* 0 cracked_hashes_AFS 0 cracked_hashes_BF 0 cracked_hashes_bfegg 0 cracked_hashes_BSDI 6 cracked_hashes_crypt 5 cracked_hashes_DES 0 cracked_hashes_DMD5 0 cracked_hashes_DOMINOSEC 0 cracked_hashes_epi 0 cracked_hashes_HDAA 0 cracked_hashes_hmac-md5 0 cracked_hashes_IPB2 0 cracked_hashes_KRB5 4 cracked_hashes_LM 1 cracked_hashes_lotus5 1 cracked_hashes_MD5 0 cracked_hashes_md5a 0 cracked_hashes_MD5-gen 0 cracked_hashes_md5ns 1 cracked_hashes_mscash 0 cracked_hashes_mssql 0 cracked_hashes_mssql05 0 cracked_hashes_MYSQL 0 cracked_hashes_mysql-fast 0 cracked_hashes_mysql-sha1 0 cracked_hashes_NETHALFLM 0 cracked_hashes_NETLM 0 cracked_hashes_NETLMv2 0 cracked_hashes_NETNTLM 0 cracked_hashes_NETNTLMv2 0 cracked_hashes_nsldap 0 cracked_hashes_NT 0 cracked_hashes_openssha 0 cracked_hashes_oracle 0 cracked_hashes_oracle11 0 cracked_hashes_phpass-md5 0 cracked_hashes_phps 0 cracked_hashes_pix-md5 0 cracked_hashes_PO 1 cracked_hashes_raw-MD5 0 cracked_hashes_raw-sha1 0 cracked_hashes_sapB 0 cracked_hashes_sapG 0 cracked_hashes_ssha 0 cracked_hashes_XSHA 19 total A mysteriously increased number of hashes? $ sort -u cracked_hashes_*|wc -l In total I have 12 different hashes. Several hashes appear to be valid for multiple hash types. $ sort cracked_hashes_*|uniq -c |grep -v "^ *1 " 2 2:10xCP/g3c1sQc 2 3:20ewVEw7R2GAw 2 4:10vrzSg43HfWI 2 6:10.6.TrD/0sas 2 7:trEJ1uz6ul/WU 2 8:$1$12$1YZMQPfPxS3.kStGKUBCO. 2 9:e10adc3949ba59abbe56e057f20f883e hese are the duplicates. e10adc3949ba59abbe56e057f20f883e appears twice, ecause it is valid for lotus5 and raw-MD5. Additionally, it has been stored in cracked_hashes_mscash as :M$9#e10adc3949ba59abbe56e057f20f883e.) And crypt recognizes all hash types supported by your system, see an 3 crypt hat means you could exclude crypt from the list of "hash formats", nless you used john --format=crypt to crack hashes not currently upported by an individual implementation (SHA-256 or SHA-512). I didn't include any SHA-256 or SHA-512 hash. fter removing cracked_hashes_crypt, these hashes remain: $ $ sort -n cracked_hashes_* :$LM$44efce164ab921ca :10xCP/g3c1sQc :20ewVEw7R2GAw :10vrzSg43HfWI :10.6.TrD/0sas :trEJ1uz6ul/WU :$1$12$1YZMQPfPxS3.kStGKUBCO. :1:$LM$e10adc3949ba59ab :2:$LM$be56e057f20f883e :e10adc3949ba59abbe56e057f20f883e :e10adc3949ba59abbe56e057f20f883e :M$9#e10adc3949ba59abbe56e057f20f883e 0:$LM$b757bf5c0d87772f Line 5 of my original cracked_hashes cracked hashes file disappeared same hash as in line 2). Line 9 produced 5 lines of "output", 4 of them unwanted: he 2 lines in cracked_hashes_LM, one in cracked_hashes_mscash, and ither the one in cracked_hashes_lotus5 or the one in racked_hashes_raw-MD5. ecause the hash e10adc3949ba59abbe56e057f20f883e appears nchanged in cracked_hashes_lotus5 and in cracked_hashes_raw-MD5, he only way to find which is the right format is: ry to crack the password again, using the contents of your original ohn.pot file. Before you continue working with john, don't forget to rename he .pot file back to john.pot. nless I misunderstood your requirement, I think there's no need o develop a new tool. On the other side, a tool could remove some of the ambiguity I entioned. But this could also be done with some more thought put nto some scripting.) egards, Frank
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.