[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Tue, 26 May 2020 11:10:02 -0500
From: Adam Lininger <arlininger@...il.com>
To: john-users@...ts.openwall.com
Subject: Re: hashcat vs. JtR
I'll share my general use in case it is helpful to you.
I usually use john for quick runs where I'm trying out a lot of
different ideas in relatively rapid succession. As start-up time becomes
a big factor in such cases, I usually run on the cpu, even if the GPU is
supported for that particular type. I also use john when I'm working
with a new hash type or when I'm having troubles.
I tend to switch to hashcat when I'm expecting multi-hour runs or any
situation that I feel I need to eek out every last ounce of performance
from the GPU's.
In short, I usually find hashcat gives me better long-run performance
but at the cost of flexibility and user friendly-ness. That said, I've
not done a comprehensive comparison of the two. I also tend to adjust
regularly on any given project as I find one to be easier or faster for
the task at hand. Otherwise, I agree entirely with what you've written.
Adam
On 5/26/20 10:36, Solar Designer wrote:
> Hi,
>
> A reporter contacted me privately asking:
>
> "In 2020, how would you judge the main differences between hashcat and
> JtR, and for which use cases?"
>
> and clarifying:
>
> "For my purposes a high-level understanding of the differences would be
> most useful"
>
> I suggested to post my thoughts on this publicly so that others can see
> and comment (and maybe correct me). My list of differences quickly got
> way too long, so I adjusted it to start with high-level comparisons and
> only then describe some detail and provide some examples. Here goes:
>
> Overall, JtR is more of a CPU tool which also supports GPUs (for some
> (non-)hashes), while hashcat is more of a GPU tool which also supports
> CPUs (for all of its supported (non-)hashes, but only through OpenCL).
>
> These days, a professional password cracking rig contains multiple GPUs,
> and hashcat is the tool to use those most efficiently. However, JtR is
> likely to also be used there, providing a more complete feature set and
> greater flexibility across the two tools combined.
>
> For casual uses, the choice will vary by personal preference,
> possibility and ease of installation on whatever system the person
> readily has, and need for higher performance and greater flexibility
> primarily on GPUs (hashcat) or CPUs (JtR).
>
> Current versions of hashcat require OpenCL (in latest release) or can
> also use CUDA without OpenCL (in git, so will be in the next release).
> This means they won't work at all (not even on CPU) without a suitable
> "driver" like this installed on the system. In contrast, JtR supports
> OpenCL, but that is optional - JtR can also be built and run without
> OpenCL on the system, in which case it will only use the CPUs. Some
> systems don't have GPUs, so hashcat's dependency on OpenCL or CUDA can
> be a nuisance on those.
>
> JtR is usually faster than hashcat on CPU (especially for slow hashes
> like bcrypt), but hashcat is usually faster than JtR on GPU (especially
> for fast hashes like NTLM). There are occasional exceptions to that.
> For example, hashcat's NTLM is impressively fast even on CPU (with
> Intel's OpenCL), while JtR's optimized md5crypt is twice faster than
> hashcat's on NVIDIA Kepler GPUs (which hashcat considers too old and
> unsupported, but in practice is able to use anyway).
>
> hashcat's multi-GPU support is much better than JtR's. hashcat
> distributes work between GPUs dynamically, whereas JtR does so before
> the attack starts. As a result, when running with a mix of different
> speed GPUs hashcat keeps all of them busy until the attack completes,
> whereas JtR may have some GPUs complete their work sooner than others.
>
> hashcat's "brain" remembers previously-tested passwords and thus lets
> multiple attacks (sequential and/or concurrent) skip repeated testing of
> the same candidate passwords. This is especially useful when a team is
> working on attacking slow (non-)hashes. JtR has no equivalent feature.
>
> While JtR is purely a password cracker, hashcat supports cracking of
> some binary keys as well.
>
> While either tool supports a few hundred of different (non-)hash types,
> there's far from a 100% overlap between what's supported by the tools.
> In cases where whatever the user needs is only supported by one of the
> tools, that dictates the choice of tool to use. The alternative to that
> is writing the missing code for the other tool (and ideally contributing
> it to there for all to use), but that requires skill and effort.
>
> For example, JtR supports cracking of passwords for FreeBSD GELI
> volumes (albeit only on CPU), which hashcat does not (at all). On the
> other hand, JtR only supports non-cascaded ciphers for Windows
> DiskCryptor (that is, when only one cipher is used at a time), whereas
> hashcat has more complete DiskCryptor support including for cascaded
> ciphers. There are many other cases of this sort. They will change
> over time, but in general nuance like this can be crucial in practice.
>
> JtR also lets the user combine fast hashes in various ways creating
> so-called "dynamic formats", even right on the command-line. These only
> work on CPU, but they do use SIMD. This is useful for cracking custom
> web applications' passwords where developers used to get creative in how
> they mix the different hash types provided by PHP or such. hashcat has
> no equivalent feature. On the other hand, for the specific combinations
> of fast hashes that hashcat does support, it supports them on GPU.
>
> There are also auxiliary tools that are developed along with one of
> these projects, but are usable with the other. For example, to attack a
> cryptocurrency wallet.dat file on GPUs, a user would first use JtR's
> bitcoin2john.py and then use hashcat on its output. Conversely,
> hashcat's 7z2hashcat.pl is the upstream project for JtR's 7z2john.pl,
> but that isn't as user-visible with 7z2john.pl being in JtR tree.
>
> Also importantly, there are differences in maximum supported password
> lengths. Recent hashcat supports lengths up to 256 for all (non-)hashes
> through inclusion of deliberately less optimal (slower) implementations
> along with optimized faster implementations that are more length-limited
> (with length limits varying by hash type, most painful perhaps being the
> optimized md5crypt's limit of 15). JtR generally only includes
> optimized implementations with length limits inherent to those
> optimizations, with only some exceptions (e.g., on CPU it includes both
> "md5crypt" with the usual limit of 15 and "md5crypt-long" capable of up
> to 125, but on GPU it currently only has the optimized "md5crypt-opencl"
> with the limit of 15). So hashcat fares much better in this respect.
> Luckily, most other optimization-related length limits are high enough
> not to pose a problem in practice (e.g., 55 for raw MD5). This issue
> doesn't apply for most modern (non-)hash types; it only exists for those
> older and not so well designed password hash types where processing time
> is significantly affected by password length.
>
> There's also not a 100% overlap in which attacks the tools can run, and
> how. JtR has many candidate password generators on CPU and also uses
> those to feed candidate passwords to GPUs, with only "mask mode"
> delegated to GPUs when cracking fast hashes like NTLM. To compensate,
> JtR is able to stack on-device mask mode on top of other modes, forming
> hybrid modes where part of the processing occurs on host CPU and part on
> GPU. This is efficient, but requires that a part of the attack be
> expressed as a mask. hashcat implements some equivalent (to JtR's) and
> some different candidate password generators, but has them implemented
> in device-side code. This makes hashcat more flexible for efficiently
> running complex attacks against fast hashes on GPU, but makes little
> difference for attacks on slow (non-)hashes.
>
> Also in terms of flexibility, JtR has more candidate password generators
> (and lets the user write custom ones in a C-like language). For
> example, its "single crack" mode derives candidate passwords from
> usernames and other user-specific information and targets them against
> the specific user's password hash (and other hashes with the same salt),
> which is efficient when cracking salted hashes. JtR's "Subsets" mode
> generates strings consisting of few different characters (progressively
> increasing that number) out of a large set of possible characters. Its
> "Keyboard" mode (one of the pre-defined custom modes) generates keyboard
> walk patterns. There are no equivalent modes built into hashcat. On
> the other hand, hashcat has "Combination" mode, which combines words
> from multiple wordlists to form candidate passphrases. JtR's closest
> built-in equivalent is PRINCE mode, kindly contributed by atom of
> hashcat. So usage of both tools can produce best results. Speaking of
> which: unfortunately, hashcat tends to greatly under-utilize GPUs when
> it's fed candidate passwords via stdin, even when the (non-)hash type is
> slow enough that such under-utilization wouldn't be inherently expected.
> A workaround is to have "john --stdout" output written out to a file and
> then read back by hashcat, but that triggers even slower startup.
>
> hashcat's startup time feels painfully long for running and adjusting
> many quick attacks manually, especially on CPU where JtR starts up
> quickly. This is in part related to hashcat's use of OpenCL even on
> CPU, but not only that: when run on a new large wordlist, hashcat spends
> a lot of time "caching" it, whereas JtR can start cracking right away.
>
> On CPU, JtR directly (without relying on an OpenCL backend) supports
> common SIMD instruction sets up to AVX-512 on x86-64.
>
> JtR also supports AltiVec on POWER and NEON/ASIMD on ARM/Aarch64. On
> the other hand, due to NVIDIA's policy of not supporting OpenCL other
> than on x86-64, JtR is unable to use NVIDIA GPUs on non-x86-64. Recent
> git hashcat regained such ability through its reintroduction of CUDA
> support, but I doubt there's a decent OpenCL implementation to use such
> non-x86 CPUs with hashcat. So if you're on a platform like this and
> want to use CPUs, use JtR. If you're on an exotic platform like this
> and want to use NVIDIA GPUs, use hashcat. For example, use JtR on a
> Raspberry Pi, but use hashcat on a NVIDIA Jetson. You can also use both
> JtR and hashcat on NVIDIA Jetson to use its CPU and GPU, respectively.
> Of course, these devices are slow, so this is for casual/hobbyist use.
>
> JtR also runs on plenty of other CPU architectures (without SIMD), for
> which there might not exist OpenCL support.
>
> JtR directly supports ZTEX 1.15y FPGA boards (including many at once).
> Unfortunately, these are now rare. Yet for those who have them, JtR
> provides very good speeds at bcrypt (higher than hashcat's on high-end
> GPUs in a per-board comparison) and at descrypt, while also supporting
> several other common hash types. hashcat could theoretically support
> FPGAs through OpenCL (and JtR could too), but this doesn't work well in
> practice (if at all).
>
> There are third-party tools that use hashcat and/or JtR. For example,
> for hashcat there's Hashtopolis: https://github.com/s3inlc/hashtopolis
> For JtR there's ByePass: https://github.com/webpwnized/byepass
> If someone wants the functionality provided by these tools, this also
> determines whether to use hashcat or JtR.
>
> There are binary packages of hashcat and/or JtR in various repositories.
> The user's operating system distribution of choice might have just one
> of those tools. Besides, it might not provide the most optimal OpenCL
> backend for licensing reasons. For example, Kali Linux appears to
> provide pocl instead of Intel's OpenCL, on which hashcat complains:
> "Not a native Intel OpenCL runtime. Expect massive speed loss." There's
> no such issue with JtR as packaged in Kali Linux.
>
> The above aspects are very far from a complete comparison of the tools.
> The examples given are just that. There can be many more such examples -
> for different (non-)hash types, different attacks, different hardware,
> different OS - where one tool or the other would have advantages.
>
> I also started writing a list of what's the same or similar between the
> tools. It quickly got way too long to include in this same message, so
> I abandoned it for now. I might try and "complete" and post it later.
>
> I'd appreciate any comments, any major differences I might have missed,
> and especially corrections if I got anything wrong.
>
> Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
