Date: Sat, 9 May 2015 05:16:24 +0300 From: Solar Designer <solar@...nwall.com> To: john-dev@...ts.openwall.com Subject: Re: Adding OpenMP support to SunMD5 On Sat, May 09, 2015 at 09:38:36AM +0800, Lei Zhang wrote: > On May 9, 2015, at 5:22 AM, Solar Designer <solar@...nwall.com> wrote: > >> Currently I manually defined _DEBUG for ease of experimenting. I'm not sure about the point of allocating those array dynamically. Is there some optimization involved? > > > > Not wasting address space. With static allocation, they'd be in .bss > > even when the SunMD5 format isn't in use. > > BTW, I don't quite understand this point. How does "wasting address space" impact program performance? It usually does not, except program startup (e.g., Kai's fuzzing might become slower). Also, you wanted these things in thread-local storage, which may be a scarce resource: http://en.wikipedia.org/wiki/Thread-local_storage "Many systems impose restrictions on the size of the thread-local memory block, in fact often rather tight limits." http://stackoverflow.com/questions/1460483/on-which-platforms-is-thread-local-storage-limited-and-how-much-is-available "There is an implementation specific limit to the number of thread specific storage objects that can be created, and this limit may be small." "[...] if you need portability, your best bet is to minimize TLS use - put a single pointer in TLS, and put everything you need in a data structure hung off that pointer." So you'd be back to dynamic allocation. > For a 64-bit build, isn't the address space big enough anyway? Not all builds are 64-bit. We actually tried to deal with .bss growth back in 2011: http://www.openwall.com/lists/john-dev/2011/11/28/1 At the time, our .bss was 140 MB. We've since got it back under control. Now it's just 9 MB. To me, this is still pretty high, but no longer ridiculous. 140 MB was a significant fraction of the 32-bit address space, unnecessarily limiting e.g. what wordlist size can be loaded to memory. Also, when someone runs --test for all formats at once, all this address space gets written to, becoming actual memory usage. We now have done() methods (we didn't back in 2011), which can (and do) free the dynamic allocations, so --test no longer consumes as much memory. You actually saw the scrypt format consume several gigabytes during --test on MIC. Now imagine this allocation was static. A few other formats could fail self-test because of it. Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.