Date: Mon, 28 Feb 2011 16:17:45 +0300 From: Solar Designer <solar@...nwall.com> To: john-users@...ts.openwall.com Subject: unique (was: GSoC 2011) Bartavelle - On Mon, Feb 28, 2011 at 11:13:02AM +0100, bartavelle wrote: > I also wrote a quick hack that helps computing that kind of huge > dictionnaries that works like "sort -u" except it doesn't sort, and is > thus MUCH faster. Isn't this exactly what JtR's "unique" program does? Perhaps you just need to increase UNIQUE_HASH_LOG and UNIQUE_BUFFER_SIZE in params.h - e.g., set them to 24 and 0x40000000, respectively, to let unique use a little over 1 GB of RAM. Have you tried that? Is your tool faster (if unique is configured to have comparable memory usage)? BTW, I made some optimizations to unique.c in what will become 1.7.7, achieving speedups of 12% to 25% on my tests (that's with default settings, without an increase in memory usage). Thank you for your feedback (including the rest of it, which I intend to comment on later; I just don't have time at the moment). Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.