[<prev] [next>] [day] [month] [year] [list]
Message-ID: <5029665E.1060102@mccme.ru>
Date: Tue, 14 Aug 2012 00:41:02 +0400
From: Alexander Cherepanov <cherepan@...me.ru>
To: john-users@...ts.openwall.com
Subject: Alexander's writeup for CMIYC 2012
Hi!
This is the first time I participated in "Crack Me If You Can"
contest and as a member of john-users team I got much fun and a ton of
experience -- thank you, guys!
= Hardware
One home computer with Intel Core2 Quad CPU Q6600 @ 2.40GHz (4 cores,
4 threads) used for wordlists manipulations and for running scripts.
Two dual-cpu servers with Intel Xeon CPU X5670 @ 2.93GHz (24 cores,
48 threads) used for actual cracking.
= Software
John the Ripper (contest edition) plus some Perl and bash scripts.
= Preparation
Fortunately I found time to make some preparations before the contest.
1. I downloaded several full wikipedia dumps -- enwiki, dewiki,
eswiki, ruwiki -- and created wordlists (sorted by frequency) from
them approximately like this:
$ bunzip2 < enwiki-20120702-pages-articles.xml.bz2 | perl -lne 'print lc $& while /\w+/g' | sort | uniq -c | sort -rn | perl -pe 's/\s*\S+\s+//' | bzip2 > enwiki.lst.bz2
(not exact command line used). Tried to do the same with Gutenberg but
it was much bigger and slower and didn't finish in time.
2. I wrote a script to generate dates in various formats and a script
to calculate salts distribution for each format. Plus some other
scripts were sketched during PHD2012 and trial pseudo-contests.
3. I chose and installed ircd on our contest server.
= Contest: challenges
Challenges were time-critical so I started by looking into
challenges. IMHO it's very important to convert source files into the
form which john can load and do it as early as possible so that all
the team can start crack them instead of fighting with converting.
john readily supported encfs, odt, pdf, rar, ssh, truecrypt and zip --
seven, not bad, eh? Well, encfs2john and odf2john didn't run with the
python from my Debian Stable, zip2john failed on contest files,
pdf2john completely failed and truecrypt2john didn't build for
me. Only rar2john and ssh2john worked right away. Not that good
overall. Though encfs2john and odf2john happened to run on Debian
Unstable. (And JimF fixed zip2john and provided dumps for zip and
pdf. Then Dhiru added support for sxc files and provided corresponding
dumps.)
After these 4 dumps were ready (and uploaded for the team) I started
to crack them with default john settings, 10 threads each. And 1h31
later happened to be the first to crack encfs-easy. These jobs were
run for ~15h and only odt-easy was additionally cracked but I wasn't
the first to do it.
Then I tried to keep track which challenges were cracked by us and by
others (according to CMIYC tweeter), and update our team on it.
= Contest: password hashes
After challenges were maxed out I switched to hashes. First of all I
tried to convert all hashes to their canonical form. This suprisingly
took much more time then it was anticipated.
Then I started to adapt my scripts for this contest, improve them,
deal with bogus lines in .pot files etc. and ended up by being in
charge of all the scripts for processing .pot files on our contest
server. That was quite nervous especially when totally switching to
improved scripts 8 hours till the end of the contest:-)
Working on scripts I had almost no time to actually crack anything.
Nevertherless I managed to run several jobs. My workflow was like
this:
- run some attack against all types of hashes with the following
script:
----------------------------------------------------------------------
#! /bin/bash
john=path/to/john
if [ "$1" = "" ]; then
echo "Usage: ./run.sh <attack-name> <john-option> ..."
echo "Example: ./run.sh enwiki-ru --wordlist=enwiki.lst --rules"
exit 1
fi
name="$1"
shift
for file in hashes-*; do
basename=`basename "$file"`
format=`echo "$basename" | cut -d . -f 2`
$john --session="$name.$basename" --pot="$name.$basename.pot" --format="$format" "$@" "$file" &
done
----------------------------------------------------------------------
and let it run in the background;
- monitor it sometimes with "top", "wc -l *.pot" and the following
script:
----------------------------------------------------------------------
#! /bin/bash
john=path/to/john
killall -HUP john
for file in *.rec; do
format=`basename "$file" .rec`
printf "%50s " "$format"
$john --status="$format"
done
----------------------------------------------------------------------
- when fast hashes finish and only 4-6 running johns remain start the
next attack;
- when time to stop an old job comes use some "ps | grep | cut | xargs kill".
I ran john with default settings, with the list of dates generated by
my script as a wordlist and with wordlists generated from wikipedia,
with and without rules. All attacks were against all hashes hence the
large overlap in cracks. The stats:
| Attacks |
format | dates default dewiki-ru enwiki enwiki-ru eswiki eswiki-ru ruwiki-ru | sum uniq val points
------------+---------------------------------------------------------------------------------+----------------------
bf | 0/23h* 0/14h* 0/08h* 0/14h* 1/31h* 0/16h* 0/08h* 0/08h* | 1 1 5000 5000
sunmd5 | 0/--- 0/--- 0/--- 0/--- 0/--- 0/--- 0/08h* 0/08h* | 0 0 3000 0
sha512crypt | 0/23h* 0/14h* 0/08h* 1/14h* 4/31h* 0/16h* 0/08h* 0/08h* | 5 4 2000 8000
mscash2 | 0/23h* 0/14h* 0/08h* 1/14h* 2/31h* 1/16h* 0/08h* 1/08h* | 5 2 600 1200
md5 | 0/23h* 4/14h* 20/08h* 44/14h* 51/31h* 31/16h* 23/08h* 15/08h* | 188 71 500 35500
dynamic_28 | 0/23h* 11/14h* 25/08h* 42/14h* 51/31h* 36/16h* 23/08h* 16/08h* | 204 76 500 38000
dynamic_21 | 1/03h 6/14h* 59/08h* 41/14h* 45/31h* 65/11h 65/08h* 57/08h* | 339 83 12 996
des | 13/01h 108/14h* 693/08h* 596/14h* 1178/31h* 685/09h 683/08h* 635/08h* | 4591 1292 9 11628
phps | 1/00h 19/14h* 111/08h* 51/08h 164/31h* 91/02h 117/08h* 106/08h* | 660 173 8 1384
salted-sha1 | 2/02h 25/14h* 170/08h* 97/14h* 230/31h* 156/07h 162/08h* 145/08h* | 987 246 8 1968
oracle11 | 2/01h 20/14h* 105/08h* 67/14h* 157/31h* 101/04h 110/08h* 101/08h* | 663 168 7 1176
mscash | 0/00h 5/14h* 86/08h* 41/06h 124/31h* 65/01h 77/08h* 69/08h* | 467 130 7 910
nt | 4/00h 722/14h* 474/00h 188/00h 630/00h 303/00h 456/00h 422/00h | 3199 1132 2 2264
raw-md5u | 0/00h 65/14h* 57/00h 20/00h 73/00h 34/00h 49/00h 45/00h | 343 117 2 234
raw-md4 | 1/00h 172/14h* 118/00h 48/00h 154/00h 69/00h 113/00h 103/00h | 778 277 1 277
raw-md5 | 1/00h 136/14h* 121/00h 59/00h 161/00h 73/00h 114/00h 109/00h | 774 233 1 233
raw-sha1 | 1/00h 272/14h* 253/00h 115/00h 353/00h 167/00h 250/00h 228/00h | 1639 523 1 523
mysql-sha1 | 1/00h 116/14h* 144/00h 56/00h 193/00h 100/00h 142/00h 126/00h | 878 261 1 261
nsldap | 1/00h 218/14h* 214/00h 85/00h 281/00h 128/00h 207/00h 175/00h | 1309 440 1 440
------------+---------------------------------------------------------------------------------+----------------------
total | | 5229 109994
The time spent on each attack is shown after the slash with the star
meaning that the attack didn't complete and was aborted.
= Experience
This contest gave me a great deal of valuable experience in password
cracking and some general sysadmin experience.
Multilayered ssh tunneling worked well with autossh. rsync over ssh
turned out to be quite nice.
OTOH ircii in gnome-terminal was quirky and hard to use leading me to
miss some messages and post garbage sometimes. My apologies for that
to the team.
= Wishlist for Korelogic
Here are several suggestions/wishes for Korelogic.
- Clarify whether there are any 8-bit passwords in the contest and how
to submit them.
- Describe how passwords for des are checked -- are 8-bit and long
variants accepted as valid or not.
- Provide feedback on which passwords were not accepted as valid.
And from the table above it seems that phps hashes are somewhat
overpriced compared to salted-sha1 and oracle11.
= Thanks
I'd like to thank Laboratory of Algebraic Geometry, HSE, and its head
Alexey Zykin personally for letting me use laboratory's servers in
this contest.
Thanks to Korelogic for good organization of the contest, to Solar
Designer for trusting me to manage server scripts and to all the team
for making the experience so fascinating!
--
Alexander Cherepanov
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
