Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Tue, 25 Jan 2011 10:46:27 -0500
Subject: Re: Password file hash type analysis in JtR

John Users/Frank/Alex,
I was thinking of this as being a pre-analysis type tool, not post cracking.  So, before any cracking begins, I would want to know which hash formats I have in the file.  This way I will make sure and manually run JtR on each of the other hash formats, if any, and not only on the first hash format JtR finds in the file.
I think the JtR executable with a command-line parameter (like: --analysis ) could do this more efficiently and cleanly, than a “For Loop”.  JtR is already doing this for the first hash format it finds, then it starts cracking.  I'd like to see it review all the hash formats, and just report which ones are contained in the file (with optional line numbers) and any it does not recognize.
Does this seem useful to you all, or just me?
-Robert B. Harris from VA


-----Original Message-----
From: Frank Dittrich <>
Sent: Tue, Jan 25, 2011 4:56 am
Subject: Re: [john-users] Password file hash type analysis in JtR

Robert Harris wrote:
 I know that Unix/Linux and Windows password files can have multiple hash
 types in them, I think the following output/utility would be useful in
What is your use case?

>>> A Unix system file with multiple hash types used and supported.  This could also have been used for
the contest held this summer.

 Potential new output:

 The following password hash format(s) was/were discovered in the input




Is that just one line describing the format?
s you would specify it on the command line (with --format=) or as john
ould report it (DES vs. Traditional DES, BF vs. OpenBSD Blowfish, ...)?

>>> Both would be nice.

> Line(s) 1-30 contained <format1> type passwords

 Line(s) 31-60 contained <format2> type passwords


 Line(s) 61-90 contained <formatx> type passwords
Without listing the corresponding hashes/passwords?
>>> Yes, I really wouldn't need the hash information, the line numbers would probably due fine.

Depending on your needs, something like this might work:
sample pot file:
cut -d: -f 1 -s john.pot|grep -n "^" > cracked_hashes
cracked_hashes would then look like this:

ename your john.pot file before continuing.
Actually, the jumbo-patched john has an option --pot=NAME,
ut I never used it... That's why I just renamed john.pot)
The next step probably requires the jumbo patch, due to --show=LEFT.
And, for our use case, it requires an empty pot file.)
for f in DES BSDI MD5 BF AFS LM; do ./john --format=$f --show=LEFT
racked_hashes > cracked_hashes_$f; done
You may also write it across multiple lines, the shell will prompt you
or the continuation:
 ./john --format=$f --show=LEFT cracked_hashes > cracked_hashes_$f
Check how the hashes are distributed according to hash type:
wc -l cracked_hashes_*
 wc -l cracked_hashes_*
 0 cracked_hashes_AFS
 0 cracked_hashes_BF
 0 cracked_hashes_BSDI
 5 cracked_hashes_DES
 4 cracked_hashes_LM
 1 cracked_hashes_MD5
10 total
On first glance, this looks good.
However, I know I included 6 DES hashes (one duplicate).
Just 5 of these appear in cracked_hashes_DES, because the duplicate
as been removed.
I'm not sure if this is unintended or not.
f a user wants to remove cracked hashes, but nevertheless
ontinue to use --single mode, keeping the duplicates might be better.)

o, where does the additional hash come from?
I included 2 LM hashes:
cracked_hashes_LM, however, contains (in addition to the expected content):
These 2 additional lines are the first and second half of the hash
10adc3949ba59abbe56e057f20f883e in line 9 of my original
racked_hashes file.
While john stores LM hashes with a preceding $LM$ marker,
he algorithm internally accepts output generated by other tools.
hat's why there's an ambiguity.)
e10adc3949ba59abbe56e057f20f883e is a raw-MD5 hash which didn't
ppear in any cracked_hashes_* file, because I just used the format list
f the unpatched john.
Using the complete list:
 for f in DES BSDI MD5 BF AFS LM NT XSHA PO raw-MD5 MD5-gen \
 IPB2 raw-sha1 md5a hmac-md5 phpass-md5 KRB5 bfegg \
 nsldap ssha openssha oracle oracle11 MYSQL \
 mysql-sha1 mscash lotus5 DOMINOSEC \
 mssql mssql05 epi phps mysql-fast pix-md5 sapG \
 sapB md5ns HDAA DMD5 crypt
 ./john --format=$f --show=LEFT cracked_hashes > cracked_hashes_$f
produces this output (to stderr):
Generic crypt(3) module: hash encoding string length 20, type id $L
ppears to be unsupported on this system; will not load such hashes.
eneric crypt(3) module: hash encoding string length 32, type id #0
ppears to be unsupported on this system; will not load such hashes.
And this list of files:
$ wc -l cracked_hashes_*
 0 cracked_hashes_AFS
 0 cracked_hashes_BF
 0 cracked_hashes_bfegg
 0 cracked_hashes_BSDI
 6 cracked_hashes_crypt
 5 cracked_hashes_DES
 0 cracked_hashes_DMD5
 0 cracked_hashes_DOMINOSEC
 0 cracked_hashes_epi
 0 cracked_hashes_HDAA
 0 cracked_hashes_hmac-md5
 0 cracked_hashes_IPB2
 0 cracked_hashes_KRB5
 4 cracked_hashes_LM
 1 cracked_hashes_lotus5
 1 cracked_hashes_MD5
 0 cracked_hashes_md5a
 0 cracked_hashes_MD5-gen
 0 cracked_hashes_md5ns
 1 cracked_hashes_mscash
 0 cracked_hashes_mssql
 0 cracked_hashes_mssql05
 0 cracked_hashes_MYSQL
 0 cracked_hashes_mysql-fast
 0 cracked_hashes_mysql-sha1
 0 cracked_hashes_NETHALFLM
 0 cracked_hashes_NETLM
 0 cracked_hashes_NETLMv2
 0 cracked_hashes_NETNTLM
 0 cracked_hashes_NETNTLMv2
 0 cracked_hashes_nsldap
 0 cracked_hashes_NT
 0 cracked_hashes_openssha
 0 cracked_hashes_oracle
 0 cracked_hashes_oracle11
 0 cracked_hashes_phpass-md5
 0 cracked_hashes_phps
 0 cracked_hashes_pix-md5
 0 cracked_hashes_PO
 1 cracked_hashes_raw-MD5
 0 cracked_hashes_raw-sha1
 0 cracked_hashes_sapB
 0 cracked_hashes_sapG
 0 cracked_hashes_ssha
 0 cracked_hashes_XSHA
19 total
A mysteriously increased number of hashes?
$ sort -u cracked_hashes_*|wc -l
In total I have 12 different hashes.
Several hashes appear to be valid for multiple hash types.
$ sort cracked_hashes_*|uniq -c |grep -v "^ *1 "
     2 2:10xCP/g3c1sQc
     2 3:20ewVEw7R2GAw
     2 4:10vrzSg43HfWI
     2 6:10.6.TrD/0sas
     2 7:trEJ1uz6ul/WU
     2 8:$1$12$1YZMQPfPxS3.kStGKUBCO.
     2 9:e10adc3949ba59abbe56e057f20f883e

hese are the duplicates.
e10adc3949ba59abbe56e057f20f883e appears twice,
ecause it is valid for lotus5 and raw-MD5.
Additionally, it has been stored in cracked_hashes_mscash as
And crypt recognizes all hash types supported by your system, see
an 3 crypt

hat means you could exclude crypt from the list of "hash formats",
nless you used john --format=crypt to crack hashes not currently
upported by an individual implementation (SHA-256 or SHA-512).
I didn't include any SHA-256 or SHA-512 hash.
fter removing cracked_hashes_crypt, these hashes remain:
$ $ sort -n cracked_hashes_*
Line 5 of my original cracked_hashes cracked hashes file disappeared
same hash as in line 2).
Line 9 produced 5 lines of "output", 4 of them unwanted:
he 2 lines in cracked_hashes_LM, one in cracked_hashes_mscash, and
ither the one in cracked_hashes_lotus5 or the one in
ecause the hash e10adc3949ba59abbe56e057f20f883e appears
nchanged in cracked_hashes_lotus5 and in cracked_hashes_raw-MD5,
he only way to find which is the right format is:
ry to crack the password again, using the contents of your original
ohn.pot file.

Before you continue working with john, don't forget to rename
he .pot file back to john.pot.

nless I misunderstood your requirement, I think there's no need
o develop a new tool.
On the other side, a tool could remove some of the ambiguity I
entioned. But this could also be done with some more thought put
nto some scripting.)

egards, Frank

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.