Date: Sun, 15 Mar 2015 21:14:48 +0300 From: Solar Designer <solar@...nwall.com> To: john-dev@...ts.openwall.com Subject: Re: [GSoC] building JtR for MIC On Sun, Mar 15, 2015 at 01:12:48PM +0800, Lei Zhang wrote: > I fixed the configuration problem and descrypt behaves normally now. > > And after deleting some files, we now have over 6GB free memory space on our MIC card. Thanks to that, scrypt can run now. Great. > This time john-jumbo ran through a bunch of benchmarks, some of which FAILED although. There are probably bugs for you to find and fix for those. While this won't matter much for actually running them on MIC - this is scalar code currently, and you'd need to switch it to MIC intrinsics later - it may matter for other uses of the same code, on other platforms. We don't want bugs sitting in there, waiting to bite someone else. One way to focus your search for bugs is to make and run a non-OpenMP build as well. Then see which formats fail self-test in both builds vs. in just one (and which one). While this won't provide conclusive results, it may provide clues and let you focus your search. > It got killed again when running smchapv2-naive. I guess it ran out of memory again? Perhaps. Can you try running the mschapv2-naive self-test on its own, not along with the rest? Something like: ./john --test --format=mschapv2-naive If it runs fine, then the problem was likely caused by memory allocation by the formats that were tested previously. Some of them intentionally allocate memory such that it can't be freed. This lowers overhead for when john is run for cracking, with just one specific format, but it results in increased total memory usage for a many-formats self-test. We should only be making such non-freed allocations for things that are small. For larger allocations, formats should free the memory in their done() methods. Perhaps some formats allocate "too much" memory (at least when the number of threads is this large) and don't free it - and perhaps you should identify them and change them to free the memory. You can watch memory usage as the self-tests are running to identify roughly when (at which format) the memory usage (or the remaining free memory) changes rapidly. You may use e.g.: [user@...er-mic0 user]$ while :; do free | grep '^-/+ buffers'; sleep 1; done -/+ buffers: 449172 7433144 -/+ buffers: 449080 7433236 -/+ buffers: 448780 7433536 > The output is a bit lengthy this time. So I put them in a text file attached below. Please see for yourself. Yes, thanks! This also shows that django-scrypt is somehow much slower than the main scrypt format. That's unexpected. But we need to merged them anyway. Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.