Date: Tue, 5 Jul 2011 12:03:57 -0500 From: "JimF" <jfoug@....net> To: <john-users@...ts.openwall.com> Subject: Re: Charset options Hi Maximilian, First off, you should become aware, that all the charset mode does, is perform a full permutation of the characters contained within the set. That is all it does. It is similar to how multiple for loops would build a string of 'words' by looping through all of the valid characters. Now, the charset mode is much 'smarter' than most. It will generate candidates based upon statistical analysis of the data which was used to build the charset. Thus, for the rockyou charset, the character patterns which make up the most frequent seen patterns in that wordlist will appear much earlier in the run. However, 'all' of the character permutations would eventually be output by the incremental engine. Now, you mentioned having passwords which are of an 'additional' pasword policy. The way which this can be done, usually is with an external rule set. The incremental engine will generate a word, and then that word is presented to the external rule filter. If the rule is satisfied, then this word is 'used', and the hash is computed. If the rule does not match, the word is not tested. The assumption here, is that the external rule processor is faster than the hash being tested for (since it DOES take time to run the filter). Now, if a word does not match the policy, it can not be a proper candidate. But if the external filter takes longer to process than processing the word, then it may be better to simply blindly run the 'incremental' code. For instance, if this were a simple MD5, and you were running this in GPU mode (cracking 100's of millions per second), then you absolutely would not want to run the words through the external filter. However, if the passwords were for salted DES, then you certainly would want to filter out words which could NEVER be a proper candidate. You mention a 'huge' performance hit. Well, is it really huge? If you are only processing 1/2 of the candidate words (or only 5%), then a 20% overall speed reduction is actually almost a 2x improvement (18x improvement if only 5% of the words pass the filter). These are the type of things you need to keep in mind. Often, it is good to run a 6 char min / 6 char max test (or 5), and see how long it takes both types to complete (with and without a filter). If it completes faster with the filter, even though the speed 'lists' to be slower, then your actual throughput is faster. As for the warning about case, it sounds like you might be doing LM testing (or some other force upcase or force low case). There is no 'way' to tell the charset (that I am aware of) to not use the a or b or c ... characters. Each of the characters is it's own beast, and has it's own rules about how likely it is, when following other characters. If you want a data set that is only of specfic characters, then you have to create it. If you are doing the rockyou and want to make an LM cracker, then I would recomend: 1. take rockyou, and chop it up into 7 character items. These are the first 7 characters of the line (or as many as possible), and then the 2nd 7 characters of the line. NOTE, the newer unique will do this for you :) using the -cut=LM switch 2. take this new file (with the chopped lines of 7 chars), and perform uppercase conversion of all text (unique's -cut=LM has already done this for you). 3. uniq this file (again unique has done this) 4. Now build a charset file using this data. Call it rockyou_lm.chr. It should only be built for 7 char max, and the up case vs low case issue has been resolved. That rockyou_lm.chr is the proper file to use (again, I am 'assuming' that you are doing LM). If this was some other format that is case insensitive, then you would need to perform the steps 1 to 4, but would have to change the 'logic' some. Possibly just upcase the file and strip any words with symbols (if the format only handles up case and numbers). Jim. ----- Original Message ----- From: "Maximilian Melcher" <melcher.maximilian@...glemail.com> > I have a small question: Is it possible to modify a charset like > rockyou.chr > to stick to a password policy? E.g. password must be 8 characters long and > must contain at least 1 letter (all upper or lower), 1 digit and 1 special > sign ? > > I've searched the FAQ/wiki and tried it with the external filters but: > > 1. I experience a huge performance inpact (~20%). As far as I understand > the > filtering is worth it because not that many words are being hashed but > limiting the charset before the generation of the words would be better, > right? But on the other hand I do not want to loose the presumption of the > charset... Any solutions for that? > > 2. John still tells me "Warning: mixed-case charset, but the current hash > type is case-insensitive; some candidate passwords may be unnecessarily > tried more than once." - how to tell the word generator? :)
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.