[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sat, 9 Sep 2006 12:58:10 +0400
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: need advice on setting up cracking system for large organisation
On Sat, Sep 09, 2006 at 04:26:06PM +1200, Russell Fulton - ISO wrote:
> Hmmm... sorry about the length of the post but I've found in the past
> that it is often better to give too much detail rather than not enough...
That's right.
> I intend to set up a (hopefully ;) secure drop box where admins can use
> scp to drop off password files to be cracked. I will then farm the
> cracking out to a bunch of 'servers' to do the actual cracking using
> 'idle' cycles on various big machines in the data centre.
This is risky.
The "secure drop box" becomes a single point of failure. You can
improve its security by actually using a dedicated box with only sshd on
it (perhaps just install Owl and not enable any other services) and by
setting a forced command in authorized_keys. (scp is no good since it
can also be used to retrieve the files back. It should not be allowed.)
What's worse, _each_ one of your cracking machines would also be a
single point of failure since its compromise might result in the
compromise of password hashes for a large number of other systems. This
risk can be mitigated a little bit by having your "secure drop box"
conceal usernames and hostnames in password files that it passes on for
cracking.
Overall, even with the improvements that I've suggested, I think that
you might be making your network less secure, not more secure.
How fast are the "big machines" when it comes to password cracking?
They might have disk arrays, large memories and caches, but this is of
no use for John the Ripper. Chances are that you can find a single
workstation that would be more suitable for the task. You would need to
secure it (or just CD-boot or install Owl and disable even sshd with
"chkconfig sshd off; service sshd stop"). Then you would pull password
files to that system and run JtR against them locally.
On many Unix systems, you can deploy pam_passwdqc:
http://www.openwall.com/passwdqc/
> One of the major issues is that we have *many* different hash formats
> out there (probably one of everything that has been used in the last 10
> years :). My understanding is that John will crack only one hash type
> at a time
Correct.
> so I need to sort incoming files by type.
Not necessarily. You can run multiple instances of JtR on all of your
files at once, with different --format=... option settings.
Essentially, you will let JtR perform this "sorting by hash type".
However, if you choose to make use of multiple machines for this, then
you might actually want to only transfer password files relevant to a
particular machine to not put other files at risk in case the machine
gets compromised.
> Is there a quick way of doing this?
>
> My initial idea is to run a single mode crack on the file as soon as it
> is received and this will find any obvious bloopers and tell me what
> type of hashes the file contains. Sensible?
Yes. It also lets you filter out hashes of the weakest passwords before
you transfer the rest to other machines.
However, multiple password hash types may be found within the same file.
In this case, John the Ripper will only autodetect whatever known hash
type it sees first. So you might have to apply your "sorting" per-line,
not per-file. (But do not run "single crack" on individual hashes - or
at least re-run it later - since it is a lot more effective when run on
entire password files - or even on multiple files at once.)
As you can see, it is a lot simpler to run multiple instances of JtR -
one per hash type - against all of your files at once. Then you do not
need to do any "manual" sorting at all.
> I then intend to farm the password out to other (faster) machines to
> crack based on word lists (yes, I've brought the CD :). From my
> understanding of how John works I'm guessing that the best strategy is
> to first group the password files by hash type and process each group in
> one go sending all files to each machine and splitting the wordlist
> between machines. Is this correct?
Yes, splitting the wordlist is more efficient than splitting password
files (of the same hash type). But wordlist-based cracking is generally
quite fast - so you might not want to distribute it across multiple
machines at all.
And you must not forget to give "incremental" mode a chance to show off.
> I have tried a run with john using the mangled list against 10 users in
> a BSD MD5 hash format password file. It took 15 hours on a fairly fast
> Intel processor.
That's because the pre-mangled wordlist is very large and the MD5-based
hashes are quite slow to compute. If you intend to be running against a
substantially larger number of hashes of this type, you can opt to use
smaller wordlists.
> Any guestimates of how this will scale with number of
> passwords? Given the 16bit salt I'm guessing that for UNIX systems it
> will be fairly linear with a slope of 1.
The FreeBSD-style MD5-based hashes typically use 48-bit salts. Yes, the
time for a JtR wordlist run should increase almost linearly with more
hashes of this type - although in practice many systems are known to
generate salts poorly, so there will likely be some duplicates despite
of the large salt size.
With the traditional DES-based hashes, things are different. These use
12-bit salts, so even with perfect salt generation you should expect
duplicate salts starting with just 100+ hashes. For 1,000 hashes, you
should expect under 900 different salts. For 10,000, it's under 3,750.
For 100,000, it's all the 4,096 possible salts.
> If this is the case should I
> send one file to each machine with the full list -- this is certainly
> easier.
Yes, from a performance point of view, it is OK to do that for slow
hashes. It might not be reasonable to do the same for faster hashes,
even when all salts are different, since with those the key setup
"overhead" is non-negligible and you would unnecessarily duplicate it.
> OTOH I assume that window hashes which I understand to be unsalted
> should scale with a slope of well under 1?
Yes, for Windows hashes (LM and NTLM), the slope for JtR run time vs.
number of hashes is close to 0.
> The initial aim of this exercise is to find the low hanging fruit that
> might be broken by brute force attack so I have generated a shorter
> mangled list with length limited to 6 characters.
I think that limiting the length is not a good way to shorten your
wordlist. Those longer candidate passwords that you're omitting in this
way are not necessarily any stronger.
Instead, you can review the list of language-specific wordlist files
that went into all.lst (this list of files is included in comments at
the start of all.lst) and generate a new combined wordlist omitting
many larger language-specific wordlists. For example, you would include
languages/English/1-tiny/* and languages/English/2-small/*, but omit
languages/English/3-large/* and languages/English/4-extra/*. Then
run it through "unique" and "john -w=new.lst -rules -stdout | unique".
> or should I try incremental for some set time/length limit?
Yes - but in addition to rather than as a replacement for wordlists.
You limit the time, not the length (leave MaxLen at 8 and let JtR decide
what lengths are optimal to try - it does so based on statistical
information found in the .chr files).
> Longer term plans are to keep hitting the files until we are fairly sure
> they conform to policy. I also intend to diff the files as they are
> resubmitted and test new passwords to the current level of the whole file.
Yes. comm(1) is the tool to use for this - after sort(1) of both
revisions, of course.
> Once this set up is up and running all systems that have password based
> services exposed to the Internet will be required to submit files at
> least once a month or when they change. Ditto for all the systems in
> the Data centre -- we have to do something useful with all those wasted
> cpu cycles :)
You might want to relax this policy for systems where you deploy
pam_passwdqc. You will need to check password files even off those
systems initially - to ensure that the policy enforcement is working as
it is supposed to - but then it might not be reasonable to put password
files off those systems at risk.
Now, this was a lengthy response. I hope it helps.
--
Alexander Peslyak <solar at openwall.com>
GPG key ID: 5B341F15 fp: B3FB 63F4 D7A3 BCCC 6F6E FC55 A2FC 027C 5B34 1F15
http://www.openwall.com - bringing security into open computing environments
Was I helpful? Please give your feedback here: http://rate.affero.net/solar
--
To unsubscribe, e-mail john-users-unsubscribe@...ts.openwall.com and reply
to the automated confirmation request that will be sent to you.
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
