Date: Mon, 3 May 2021 19:53:42 +0200 From: Solar Designer <solar@...nwall.com> To: john-users@...ts.openwall.com Subject: Re: source of information for John's charset files On Mon, May 03, 2021 at 04:18:15PM +0200, Solar Designer wrote: > On Sun, May 02, 2021 at 11:00:34PM -0400, Matt Weir wrote: > > Side note, I just saw your most recent results of training/running against > > RockYou. I'm willing to admit I'm wrong if you are getting better results > > training without dupes. That's just contrary to what I've seen in the past. > > I might need to run some tests of my own to look into this. > > Note: better results when the test set is also without dupes. However, > I think that's what matters after most dupes are eliminated using a > wordlist anyway in real-world usage of our tools. I found an easily publicly available example where training without dupes produces worse results even when the test set is also without dupes. Running through RockYou as a wordlist first corrects that. I took phpbb-withmd5.txt.bz2 from here: https://wiki.skullsecurity.org/Passwords phpbb phpbb.txt.bz2 (868,606 bytes) n/a 2009-01 Ordered by commonness Cracked from md5 by Brandon Enright (97%+ coverage) phpbb with count phpbb-withcount.txt.bz2 (872,867 bytes) n/a phpbb with md5 phpbb-withmd5.txt.bz2 (4,117,887 bytes) n/a My 3 training sets are: RockYou with dupes, RockYou unique (full 14.3M this time), HIBP v7 cracked excluding fbobh_* (458M). Cracked phpbb uniques with incremental mode at 1 billion candidates are: RockYou with dupes - 34.7% RockYou unique - 33.6% HIBP v7 cracked - 30.9% RockYou wordlist - 39.3% RockYou wordlist together with: RockYou with dupes - 51.3% RockYou unique - 51.5% HIBP v7 cracked - 50.6% RockYou wordlist with best64 rules (only, no incremental): RockYou + best64 - 56.3% RockYou wordlist with best64 rules (above) together with the incremental runs (first set of results above): RockYou with dupes - 60.4% RockYou unique - 60.5% HIBP v7 cracked - 60.1% All of these combined - 62.3%. Of course, real attacks on fast hashes like this would test many more candidates and get almost all hashes cracked. These simulations I run are to see how effective the approaches are in case similar passwords were used with slow (non-)hashes and/or with many different salts. Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.