Date: Fri, 18 Sep 2015 19:57:05 +0300 From: Solar Designer <solar@...nwall.com> To: john-dev@...ts.openwall.com Subject: current large hash file speeds and bottlenecks (was: ldr_split_line() performance regression) On Tue, Sep 15, 2015 at 09:29:35PM -0700, Fred Wang wrote: > That is a *whole* lot better than when you started. I'm still interested in seeing what happens with --fork=32 on the larger machines, but - wow! Well done! Thanks. For the currently committed code (which isn't my latest, and is a bit slower than what I run on 2x E5420 now) and the currently committed john.conf defaults, I get on 2x E5-2670 v1: --fork=32: real 0m17.399s user 2m38.080s sys 0m19.943s --fork=16: real 0m19.137s user 1m59.104s sys 0m9.040s --fork=8: real 0m23.691s user 1m46.217s sys 0m4.615s Loading takes a little over 10 seconds on one core, the rest is cracking. This machine is supposed to be slower than the 2x E5-2680 v2 that you reported similar speeds for. This is a non-OpenMP build ("./configure --disable-openmp") made with: gcc version 4.9.1 20140922 (Red Hat 4.9.1-10) (GCC) (from devtoolset-3 for Scientific Linux 6), and the command line is: time ./john -form=raw-md5 -w=10m.pass -ru=fred-best64 -nolog -v=1 -save-mem=1 -fork=32 29m.txt 1709703 passwords are cracked in all cases, like before. The next hurdle is "--show", which is still unreasonably slow: takes several minutes to display those same passwords we've cracked in ~20 seconds. I haven't looked into it yet. I think this might be a regression with some changes in jumbo as I seem to recall "--show" working no slower than normal loading of large files before. I hope it's something trivial. After that, speeding up loading of large salted hash databases, which is hopefully mostly about tuning of SALT_HASH_LOG. And maybe we need to have john print one-line tips each time it's run on a terminal, like e.g. radare2 does. Suggest options like "-nolog -v=1 -save-mem=1", randomly or when it feels a given run would likely benefit from them. Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.