Date: Sat, 28 Jan 2012 00:48:35 +0400 From: Solar Designer <solar@...nwall.com> To: john-dev@...ts.openwall.com Subject: unique using more than 2 GB RAM magnum, Jim - Here's a wish list item: We should make "unique" able to use more than 2 GB of memory. As it turns out, "unique" at 2 GB is about twice slower than "sort -u -S 14G" (on a 16 GB RAM machine), although of course this may vary by input data. Maybe "unique" should start using 40-bit offsets (good for up to 1 TB of RAM). I am concerned that it will become slightly less efficient (in terms of both speed and memory usage) when this new functionality is not being made use of, though. Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.