Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sun, 21 Jul 2013 10:22:31 +0530
From: Sayantan Datta <>
Subject: Re: md5 hash comparisons


On Sun, Jul 21, 2013 at 4:54 AM, Solar Designer <> wrote:

> Yes, for 16M you need global memory - even the multiple bitmaps or Bloom
> filter approach in local memory doesn't work at all.  (Maybe we should
> include a not-data-dependent conditional branch to skip the local
> bitmaps and go straight to global memory when the number of loaded
> hashes is this large.  We'd need to tune the threshold.)

Yes, this is what I have been thinking. Currently there are 2x2KB bitmaps
followed by two sequential search in the md5 kernel. It works well for
smaller number of hashes but for large number of hashes the two bitmaps
will most likely fail every time. Regarding bitmap in global memory, even
though it allows us to uniquely identify all hashes using 4*512MB bitmaps
but one major drawback with bitmaps is that it turns sequential search into
random access which will most likely cause bank conflicts and lower
performance. I tried one 256MB bitmap in the md5 kernel but the performance
wasn't good. For large number of hashes the hashtable  approach might be


Content of type "text/html" skipped

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.