Date: Fri, 26 Apr 2013 02:31:47 +0400 From: Solar Designer <solar@...nwall.com> To: john-dev@...ts.openwall.com Subject: revised incremental mode and charset files (was: Bleeding-jumbo branch updated from core) On Thu, Apr 25, 2013 at 10:45:50PM +0200, magnum wrote: > I have merged latest commits from core CVS into the bleeding git branch. Thank you! I've just pushed another related change out to the public CVS repository: speeding up charset file generation, as this becomes more important with increased CHARSET_* settings that we're going to use. There were two commits: "Optimized the charset file generation algorithms and code ("--make-charset"). On a certain test case, this has improved the time to generate a charset file from 55.5 to 19.5 seconds while generating the exact same file." and: "In cmp_ratio(), stabilize the sorting order differently from the way we happened to be doing it before. This results in subtly different charset files being generated for the same input data, but arguably the new stabilization is more logical, and it helps reduce the number of recalculations and thus speeds up charset file generation (now 16.5 seconds in the test case mentioned in the previous commit)." The mentioned test case is full RockYou (including dupes), CHARSET_MAX 0xFF, and CHARSET_LENGTH 15 (I also tested 20 and 24, with just slightly longer charset file generation times with the new code). The times are as measured on bull. > Worse, the bleeding branch now does not contain any working charset files for Incremental, because the old ones are incompatible (at least for now). Try creating new ones from the Rockyou list, for testing stuff. > > Obviously, with [current] latest bleeding version you can't resume an Incremental session started with an earlier version. Yes. I considered keeping/introducing some backwards compatibility code, like we had for JtR 1.6's (not a typo) incremental mode revision until now. This would be reasonably easy if we kept CHARSET_MAX and CHARSET_LENGTH intact, but clearly we're not going to. Since at least the default CHARSET_LENGTH will change, which would break compatibility with old charset files even without code changes, my options were either to revise/add a lot of code to support charset files with arbitrary CHARSET_LENGTH settings (different from the current build's) or to drop the backwards compatibility altogether and simply standardize on new and much increased default CHARSET_LENGTH (and maybe new CHARSET_MAX as well). I chose the latter. What do you think the new CHARSET_LENGTH and CHARSET_MAX defaults should be? I think the minimum for new defaults is CHARSET_LENGTH 15 (increased from the current/old default of 8) and CHARSET_MAX 0x7E (same as it is now), but should we go higher? If we support 8-bit chars by default (increase CHARSET_MAX), then should the supplied .chr files include those or not? If yes, then where should the input data come from? RockYou has about 18k strings with 8-bit chars, but it is unclear how many of those are actual passwords; I think the percentage of junk is a lot higher within this portion of RockYou than it is in RockYou overall. Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.