Date: Sat, 20 Mar 2010 01:19:31 +0300 From: Solar Designer <solar@...nwall.com> To: john-users@...ts.openwall.com Subject: Re: avoid duplicates with rules On Fri, Mar 19, 2010 at 07:57:31PM +0100, websiteaccess@...il.com wrote: > I have in my wordlist 2 words : > justine > justone [...] > my john.conf is now > > $1 > /i si[aeouy]$[0-9] > > As you can see, the 2 differents rules will generate 2 times the same word Yes, "justone1" will be generated twice - the first time by applying the "$1" rule to "justone" and the second time by applying the "/i sio $1" rule to "justine". > I want keep both rules, how can I avoid duplicate words ? You can't. Well, if you really have to (e.g., if your hashes are of a slow to compute type), you'll have to run "--stdout" through "unique". > There is a special command in JTR's rule to avoid duplicates, but i'm > not very comfortable with these. There are many rule commands that are not "special", but that can be used to avoid or reduce duplicates. However, this only works for duplicates that would result from application of different rules to the same input word, not for those resulting from similar input words. For this reason, the default rulesets supplied with JtR _do_ produce some duplicates on some wordlists, although they would have produced many more duplicates if they did not try to avoid those resulting from application of different rules to the same input word (they do). For example, the default wordlist mode ruleset of JtR 1.7.5 run against password.lst included with that version of JtR with "--stdout" (and no length limit) produces 141,273 candidate passwords, of which only 133,871 are unique (that's 94.8%). However, converting password.lst to all-lowercase and then sorting it alphabetically reduces these numbers to 134,476 and 133,867, respectively (99.5% unique), which is why such conversion is recommended for large and not-so-targeted wordlists. Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.