Date: Sat, 9 Sep 2006 12:58:10 +0400 From: Solar Designer <solar@...nwall.com> To: john-users@...ts.openwall.com Subject: Re: need advice on setting up cracking system for large organisation On Sat, Sep 09, 2006 at 04:26:06PM +1200, Russell Fulton - ISO wrote: > Hmmm... sorry about the length of the post but I've found in the past > that it is often better to give too much detail rather than not enough... That's right. > I intend to set up a (hopefully ;) secure drop box where admins can use > scp to drop off password files to be cracked. I will then farm the > cracking out to a bunch of 'servers' to do the actual cracking using > 'idle' cycles on various big machines in the data centre. This is risky. The "secure drop box" becomes a single point of failure. You can improve its security by actually using a dedicated box with only sshd on it (perhaps just install Owl and not enable any other services) and by setting a forced command in authorized_keys. (scp is no good since it can also be used to retrieve the files back. It should not be allowed.) What's worse, _each_ one of your cracking machines would also be a single point of failure since its compromise might result in the compromise of password hashes for a large number of other systems. This risk can be mitigated a little bit by having your "secure drop box" conceal usernames and hostnames in password files that it passes on for cracking. Overall, even with the improvements that I've suggested, I think that you might be making your network less secure, not more secure. How fast are the "big machines" when it comes to password cracking? They might have disk arrays, large memories and caches, but this is of no use for John the Ripper. Chances are that you can find a single workstation that would be more suitable for the task. You would need to secure it (or just CD-boot or install Owl and disable even sshd with "chkconfig sshd off; service sshd stop"). Then you would pull password files to that system and run JtR against them locally. On many Unix systems, you can deploy pam_passwdqc: http://www.openwall.com/passwdqc/ > One of the major issues is that we have *many* different hash formats > out there (probably one of everything that has been used in the last 10 > years :). My understanding is that John will crack only one hash type > at a time Correct. > so I need to sort incoming files by type. Not necessarily. You can run multiple instances of JtR on all of your files at once, with different --format=... option settings. Essentially, you will let JtR perform this "sorting by hash type". However, if you choose to make use of multiple machines for this, then you might actually want to only transfer password files relevant to a particular machine to not put other files at risk in case the machine gets compromised. > Is there a quick way of doing this? > > My initial idea is to run a single mode crack on the file as soon as it > is received and this will find any obvious bloopers and tell me what > type of hashes the file contains. Sensible? Yes. It also lets you filter out hashes of the weakest passwords before you transfer the rest to other machines. However, multiple password hash types may be found within the same file. In this case, John the Ripper will only autodetect whatever known hash type it sees first. So you might have to apply your "sorting" per-line, not per-file. (But do not run "single crack" on individual hashes - or at least re-run it later - since it is a lot more effective when run on entire password files - or even on multiple files at once.) As you can see, it is a lot simpler to run multiple instances of JtR - one per hash type - against all of your files at once. Then you do not need to do any "manual" sorting at all. > I then intend to farm the password out to other (faster) machines to > crack based on word lists (yes, I've brought the CD :). From my > understanding of how John works I'm guessing that the best strategy is > to first group the password files by hash type and process each group in > one go sending all files to each machine and splitting the wordlist > between machines. Is this correct? Yes, splitting the wordlist is more efficient than splitting password files (of the same hash type). But wordlist-based cracking is generally quite fast - so you might not want to distribute it across multiple machines at all. And you must not forget to give "incremental" mode a chance to show off. > I have tried a run with john using the mangled list against 10 users in > a BSD MD5 hash format password file. It took 15 hours on a fairly fast > Intel processor. That's because the pre-mangled wordlist is very large and the MD5-based hashes are quite slow to compute. If you intend to be running against a substantially larger number of hashes of this type, you can opt to use smaller wordlists. > Any guestimates of how this will scale with number of > passwords? Given the 16bit salt I'm guessing that for UNIX systems it > will be fairly linear with a slope of 1. The FreeBSD-style MD5-based hashes typically use 48-bit salts. Yes, the time for a JtR wordlist run should increase almost linearly with more hashes of this type - although in practice many systems are known to generate salts poorly, so there will likely be some duplicates despite of the large salt size. With the traditional DES-based hashes, things are different. These use 12-bit salts, so even with perfect salt generation you should expect duplicate salts starting with just 100+ hashes. For 1,000 hashes, you should expect under 900 different salts. For 10,000, it's under 3,750. For 100,000, it's all the 4,096 possible salts. > If this is the case should I > send one file to each machine with the full list -- this is certainly > easier. Yes, from a performance point of view, it is OK to do that for slow hashes. It might not be reasonable to do the same for faster hashes, even when all salts are different, since with those the key setup "overhead" is non-negligible and you would unnecessarily duplicate it. > OTOH I assume that window hashes which I understand to be unsalted > should scale with a slope of well under 1? Yes, for Windows hashes (LM and NTLM), the slope for JtR run time vs. number of hashes is close to 0. > The initial aim of this exercise is to find the low hanging fruit that > might be broken by brute force attack so I have generated a shorter > mangled list with length limited to 6 characters. I think that limiting the length is not a good way to shorten your wordlist. Those longer candidate passwords that you're omitting in this way are not necessarily any stronger. Instead, you can review the list of language-specific wordlist files that went into all.lst (this list of files is included in comments at the start of all.lst) and generate a new combined wordlist omitting many larger language-specific wordlists. For example, you would include languages/English/1-tiny/* and languages/English/2-small/*, but omit languages/English/3-large/* and languages/English/4-extra/*. Then run it through "unique" and "john -w=new.lst -rules -stdout | unique". > or should I try incremental for some set time/length limit? Yes - but in addition to rather than as a replacement for wordlists. You limit the time, not the length (leave MaxLen at 8 and let JtR decide what lengths are optimal to try - it does so based on statistical information found in the .chr files). > Longer term plans are to keep hitting the files until we are fairly sure > they conform to policy. I also intend to diff the files as they are > resubmitted and test new passwords to the current level of the whole file. Yes. comm(1) is the tool to use for this - after sort(1) of both revisions, of course. > Once this set up is up and running all systems that have password based > services exposed to the Internet will be required to submit files at > least once a month or when they change. Ditto for all the systems in > the Data centre -- we have to do something useful with all those wasted > cpu cycles :) You might want to relax this policy for systems where you deploy pam_passwdqc. You will need to check password files even off those systems initially - to ensure that the policy enforcement is working as it is supposed to - but then it might not be reasonable to put password files off those systems at risk. Now, this was a lengthy response. I hope it helps. -- Alexander Peslyak <solar at openwall.com> GPG key ID: 5B341F15 fp: B3FB 63F4 D7A3 BCCC 6F6E FC55 A2FC 027C 5B34 1F15 http://www.openwall.com - bringing security into open computing environments Was I helpful? Please give your feedback here: http://rate.affero.net/solar -- To unsubscribe, e-mail john-users-unsubscribe@...ts.openwall.com and reply to the automated confirmation request that will be sent to you.
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.