Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Fri, 18 Sep 2015 19:57:05 +0300
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: current large hash file speeds and bottlenecks (was: ldr_split_line() performance regression)

On Tue, Sep 15, 2015 at 09:29:35PM -0700, Fred Wang wrote:
> That is a *whole* lot better than when you started.  I'm still interested in seeing what happens with --fork=32 on the larger machines, but - wow!  Well done!

Thanks.  For the currently committed code (which isn't my latest, and is
a bit slower than what I run on 2x E5420 now) and the currently
committed john.conf defaults, I get on 2x E5-2670 v1:

--fork=32:

real    0m17.399s
user    2m38.080s
sys     0m19.943s

--fork=16:

real    0m19.137s
user    1m59.104s
sys     0m9.040s

--fork=8:

real    0m23.691s
user    1m46.217s
sys     0m4.615s

Loading takes a little over 10 seconds on one core, the rest is cracking.

This machine is supposed to be slower than the 2x E5-2680 v2 that you
reported similar speeds for.

This is a non-OpenMP build ("./configure --disable-openmp") made with:

gcc version 4.9.1 20140922 (Red Hat 4.9.1-10) (GCC)

(from devtoolset-3 for Scientific Linux 6), and the command line is:

time ./john -form=raw-md5 -w=10m.pass -ru=fred-best64 -nolog -v=1 -save-mem=1 -fork=32 29m.txt

1709703 passwords are cracked in all cases, like before.

The next hurdle is "--show", which is still unreasonably slow: takes
several minutes to display those same passwords we've cracked in ~20
seconds.  I haven't looked into it yet.  I think this might be a
regression with some changes in jumbo as I seem to recall "--show"
working no slower than normal loading of large files before.  I hope
it's something trivial.

After that, speeding up loading of large salted hash databases, which is
hopefully mostly about tuning of SALT_HASH_LOG.

And maybe we need to have john print one-line tips each time it's run on
a terminal, like e.g. radare2 does.  Suggest options like "-nolog -v=1
-save-mem=1", randomly or when it feels a given run would likely benefit
from them.

Alexander

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.