Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Sat, 28 Jan 2012 00:48:35 +0400
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: unique using more than 2 GB RAM

magnum, Jim -

Here's a wish list item:

We should make "unique" able to use more than 2 GB of memory.  As it
turns out, "unique" at 2 GB is about twice slower than "sort -u -S 14G"
(on a 16 GB RAM machine), although of course this may vary by input
data.  Maybe "unique" should start using 40-bit offsets (good for up to
1 TB of RAM).  I am concerned that it will become slightly less
efficient (in terms of both speed and memory usage) when this new
functionality is not being made use of, though.

Alexander

Powered by blists - more mailing lists

Your e-mail address:

Powered by Openwall GNU/*/Linux - Powered by OpenVZ