[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Wed, 11 Jul 2012 13:33:06 +0200
From: Frank Dittrich <frank_dittrich@...mail.com>
To: john-users@...ts.openwall.com
Subject: Re: our own training pseudo contest before CMIYC 2012
On 07/11/2012 10:35 AM, Solar Designer wrote:
> Frank, Simon, Aleksey, all -
>
> On Tue, Jul 10, 2012 at 05:48:47PM +0200, Frank Dittrich wrote:
>> On 07/10/2012 02:28 PM, Aleksey Cherepanov wrote:
>>> As you know there will be Crack Me If You Can contest on July 26-29
>>> (or like). But it would be nice to make a training contest before it
>>> to prepare ourself.
>>
>> Wouldn't other preparations for the real contest more important?
>
> Maybe, or maybe not, but one reason why I suggested doing this pseudo
> contest to Aleksey is that the real contest would be a wrong time to
> test out his MJohn tool for the very first time.
That's correct. Trying to use MJohn for the very first time during the
real contest certainly wouldn't work.
> Our priority in the
> real contest should be getting the hashes cracked, not learning a new
> tool. For example, in the real contest, unless we choose to focus
> solely on our own stuff rather than on achieving a good score, I would
> not even look at MJohn, unless I already tried it out in a similar
> setting by that time and liked it (or at least did not hate it much).
> I think it will/should be similar for others.
Right. Will Aleksey have something ready which can be tested about a
week before the contest starts?
Even if he has.
I think for the real contest, there will be more users than in the
training pseudo contest. (But may be many users can afford to spend
those few hours one week before the real contest, I don't know.)
> In fact, I have my doubts regarding how many of us would actually use
> MJohn during the contest even if we try to practice with it in advance,
> but at least it would have a chance then. Without this practice, it has
> no chance at all. And without being tested in a real contest, MJohn
> would also have little chance to evolve in a relevant direction later.
I think this is correct.
Even if we test it in a pseudo contest, we have to be able to integrate
results of users who don't use MJohn during the real contest.
At least, we need to support users who just want to drop their john.pot
files (or deltas compared to previous submissions or deltas compared to
the current "central/global" john.pot on the server).
>> May be the CMIYC 2012 hashes require conversion of hashes into john
>> specific formats, as the PHDays Hash Runner files did.
>
> What does this mean for our contest preparations?
> I see no action item here yet.
My idea was using something like
./john --format=$f --show=LEFT input > $f.uncracked
for all formats john supports, may be ignoring "duplicate" formats which
accept the same input/generate the same canonical hash representations.
One problem with this approach is that ambiguous hashes would end up in
different *.uncracked files.
Another problem is that using --show=LEFT in its current implementation
has some drawbacks, at least if you still want to use --single mode or
if you want to be able to use --users= or --groups=.
>> We also still need some easy way to create input files with uncracked
>> hashes only, but not suppressing duplicate lines (as --show=LEFT does.
>
> FYI, scripts I used on our contest server during CMIYC 2010 and 2011 did
> not use --show=LEFT, but rather processed john.pot files directly.
IIRC, we were lucky that the input files already used the preferred
canonical hash representation.
I imagine doing this with scripts will be harder if there are multiple
valid representations for the same hash, and just one of those
representations is used for john.pot lines (not necessarily the one that
is found in the input files).
> Unfortunately, this fails for LM hashes (where --show is needed to merge
> the halves), but for other contest hashes it worked fine, I think.
> Maybe --show=LEFT would as well. I must admit I don't see how the issue
> with suppressing duplicate lines is relevant. Can you explain, maybe
> based on some contest-relevant example?
I don't know if this is relevant for the contest.
I mentioned the problem here:
http://openwall.com/lists/john-users/2012/05/06/1
$ cat passwd
user1:XXxzOu6maQKqQ:1001:1001:user 1:/home/user1:/bin/bash
user2:XXxzOu6maQKqQ:1002:1002:user 2:/home/user2:/bin/bash
$ ./john --show=LEFT passwd
user1:XXxzOu6maQKqQ
If I still want to use single mode for the uncracked hashes, I'd prefer
to keep as many different user names / GECOS fields as possible.
Also, if some of the accounts with uncracked passwords are admin
accounts (identified by uid/gid), it is unfortunate that this
information is lost after using --show=LEFT
>> I am sure there are other tasks which can be identified by looking at
>> what prevented us from doing better at CMIYC 2012 or PHDays Hash Runner.
>> And we might need to check and polish some scripts we used during the
>> CMIYC contest, like the ones for validating new pot file entries...
>
> Of course, there are many other tasks. This does not mean that a trial
> contest is not a good task. Every time a real contest starts, we find
> that we need to create accounts for new people joining, we have
> coordination issues, we have bugs in recent revisions of our tools (not
> tested enough yet), etc. I think having a trial contest shortly before
> the real one (just a few days earlier) will minimize such issues and
> will also help us fix bugs in cutting-edge JtR code, including in
> recently-added GPU code.
Yes, that's right.
How easy is creating a GPU build for an average user?
(I didn't look at the how-tos (wiki. doc/*) for quite some time, so I
have no idea.)
Could we prepare a linux image with a pre-built john for GPU for 64bit
ubuntu, which can be copied onto a 16GB USB stick (or even 8 GB, if
that's considered big enough)?
This would help users who only know Windows, but are willing to try out
something else. They don't need to know how to install Linux, how to
build john with GPU support, how to resize a Linux partition.
They only need to know how to boot their machine from an USB stick.
And they might need an easy to follow instruction on how to change the
keyboard layout.
But may be there are license issues, and we could only distribute shell
scripts which the user has to execute to download the binary drivers and
additional software.
>> May be we also need to look at thing competing tools can do better than
>> john, and how we can compensate for it:
>> -hashcat's mask mode
>> -hash cats ability to generate password candidates from two separate
>> inputs (left side + some rule / right side + some rule), where
>> left/right side can either be a word list file or a mask (say ddds or
>> whatever the syntax is for "3 digits, followed by a special character")
>> -...?
>
> I don't mind. For processing 2+ separate inputs, I'd currently use my
> Perl scripts to preprocess wordlists, although it does make sense to get
> similar functionality into John proper eventually (not for this contest).
Are those scripts located on the contest server?
>> Also, IIRC, hashcat recently had a contest to find "best 64" rules.
>> May be we can prepare something similar.
>> Use the default password.lst, converted to lower case, removing any
>> resulting duplicates, but adding "rockyou".
>> Then, try to find which rules in which sequence would be the best to
>> crack dummy hashes generated from the rockyou password list.
>
> This is a good idea, and I think Simon is the one to organize this
> contest for us. ;-) However, I think it is not sufficiently relevant to
> CMIYC 2012 specifically.
You are probably right. The contest hashes will most likely need to be
attacked using a different approach.
> Summary: I support Aleksey's request/invitation to join our team for a
> trial contest. I think it can be 4 hours on July 21 and then maybe
> another 4 hours for a second try on July 22 (leaving time between these
> two days e.g. for Aleksey to fix MJohn bugs and for any of us to fix
> John bugs). The time of day should be chosen such that it'd be evening /
> early night in Europe and western Russia, and morning in Americas.
> For example, the pseudo contest could run from 17:00 UTC to 21:00 UTC.
>
> What do you think?
OK, I'll make sure I have time to participate.
Frank
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
