john-users - Re: JTR not able to crack some hashes utf-8 ?

Follow @Openwall on Twitter for new release announcements and other news

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date: Tue, 15 Dec 2009 13:53:42 +0100
From: SL <auditor@...chat.de>
To: john-users@...ts.openwall.com
Subject: Re: JTR not able to crack some hashes utf-8 ?

Am 2009-12-14 um 22:13 schrieb websiteaccess:

> On Sun, 13 Dec 2009 23:02:17 +0300, Solar Designer wrote:
>> Most likely, your terminal settings did not match the encoding/ 
>> charset
>> that your wordlist and passwords were using.
>
> I need more help :(
>
> My browser has UTF-8 encoding, I went to
> http://www.insidepro.com/hashes.php?lang=eng   then
> convert knüpp to MD5 ( knüpp = 4cd6830f5a766cd334145eacce13e764 )

The InsidePro Website uses "windows-1251" encoding.

# curl -s "http://www.insidepro.com/hashes.php?lang=eng" | grep  
"Content-Type"
<meta http-equiv="Content-Type" content="text/html;  
charset=windows-1251">

I  don't know what happens when you enter UTF-8 characters into a  
windows-1251 HTML form, but probably not what you expect.
In this case, you get the umlaut converted into HTML character  
encoding (&#252;).

>  I put and save user:4cd6830f5a766cd334145eacce13e764   in a texte  
> with
> UTF-8 encoding
>
>  I put word knüpp in a texte file with the same encoding (UTF-8)
>
>  I had configured my Terminal with UTF-8 encoding.
>
>  I did, ./john -format=raw-md5 -w:mydico_utf8.txt hash_utf-8.txt
>  [...]
>  All is UTF-8 encoded , why Jtr can't crack this hash ???

Your hash is NOT md5(utf8('knüpp')). That's why. md5(utf8('knüpp'))  
is 92c88d6d7741b3ad662a91e8119776a1:

# echo -n "knüpp" | iconv -t utf8 | openssl md5
92c88d6d7741b3ad662a91e8119776a1
# echo -n "knüpp" | iconv -t latin1 | openssl md5
6be78ddecc551b3cb0d78402934a193b

Now check this:

# echo -n "kn&#252;pp" | openssl md5
4cd6830f5a766cd334145eacce13e764

There's your hash. ;-)

If you want it to get cracked, you should prepare your wordlist  
accordingly (presuming your terminal is UTF-8):

# echo "knüpp" > wordlist-utf8
# iconv -f utf8 -t latin1 wordlist-utf8 > wordlist-latin1
# iconv -f utf8 -t ascii//TRANSLIT > wordlist-ascii
# cat wordlist-utf8 | php -r "print preg_replace('/[^[:alnum:] 
[:space:][:punct:]]/ue', 'chr(38).chr(35).ord(\$0).chr(59)',  
file_get_contents('php://stdin'));" > wordlist-htmlent
# cat wordlist-* | ./unique wordlist

Then "./john -format=raw-md5 -w:wordlist hash_utf-8.txt" should crack  
your hash.

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.