Date: Tue, 26 May 2020 17:36:24 +0200 From: Solar Designer <solar@...nwall.com> To: john-users@...ts.openwall.com Subject: hashcat vs. JtR Hi, A reporter contacted me privately asking: "In 2020, how would you judge the main differences between hashcat and JtR, and for which use cases?" and clarifying: "For my purposes a high-level understanding of the differences would be most useful" I suggested to post my thoughts on this publicly so that others can see and comment (and maybe correct me). My list of differences quickly got way too long, so I adjusted it to start with high-level comparisons and only then describe some detail and provide some examples. Here goes: Overall, JtR is more of a CPU tool which also supports GPUs (for some (non-)hashes), while hashcat is more of a GPU tool which also supports CPUs (for all of its supported (non-)hashes, but only through OpenCL). These days, a professional password cracking rig contains multiple GPUs, and hashcat is the tool to use those most efficiently. However, JtR is likely to also be used there, providing a more complete feature set and greater flexibility across the two tools combined. For casual uses, the choice will vary by personal preference, possibility and ease of installation on whatever system the person readily has, and need for higher performance and greater flexibility primarily on GPUs (hashcat) or CPUs (JtR). Current versions of hashcat require OpenCL (in latest release) or can also use CUDA without OpenCL (in git, so will be in the next release). This means they won't work at all (not even on CPU) without a suitable "driver" like this installed on the system. In contrast, JtR supports OpenCL, but that is optional - JtR can also be built and run without OpenCL on the system, in which case it will only use the CPUs. Some systems don't have GPUs, so hashcat's dependency on OpenCL or CUDA can be a nuisance on those. JtR is usually faster than hashcat on CPU (especially for slow hashes like bcrypt), but hashcat is usually faster than JtR on GPU (especially for fast hashes like NTLM). There are occasional exceptions to that. For example, hashcat's NTLM is impressively fast even on CPU (with Intel's OpenCL), while JtR's optimized md5crypt is twice faster than hashcat's on NVIDIA Kepler GPUs (which hashcat considers too old and unsupported, but in practice is able to use anyway). hashcat's multi-GPU support is much better than JtR's. hashcat distributes work between GPUs dynamically, whereas JtR does so before the attack starts. As a result, when running with a mix of different speed GPUs hashcat keeps all of them busy until the attack completes, whereas JtR may have some GPUs complete their work sooner than others. hashcat's "brain" remembers previously-tested passwords and thus lets multiple attacks (sequential and/or concurrent) skip repeated testing of the same candidate passwords. This is especially useful when a team is working on attacking slow (non-)hashes. JtR has no equivalent feature. While JtR is purely a password cracker, hashcat supports cracking of some binary keys as well. While either tool supports a few hundred of different (non-)hash types, there's far from a 100% overlap between what's supported by the tools. In cases where whatever the user needs is only supported by one of the tools, that dictates the choice of tool to use. The alternative to that is writing the missing code for the other tool (and ideally contributing it to there for all to use), but that requires skill and effort. For example, JtR supports cracking of passwords for FreeBSD GELI volumes (albeit only on CPU), which hashcat does not (at all). On the other hand, JtR only supports non-cascaded ciphers for Windows DiskCryptor (that is, when only one cipher is used at a time), whereas hashcat has more complete DiskCryptor support including for cascaded ciphers. There are many other cases of this sort. They will change over time, but in general nuance like this can be crucial in practice. JtR also lets the user combine fast hashes in various ways creating so-called "dynamic formats", even right on the command-line. These only work on CPU, but they do use SIMD. This is useful for cracking custom web applications' passwords where developers used to get creative in how they mix the different hash types provided by PHP or such. hashcat has no equivalent feature. On the other hand, for the specific combinations of fast hashes that hashcat does support, it supports them on GPU. There are also auxiliary tools that are developed along with one of these projects, but are usable with the other. For example, to attack a cryptocurrency wallet.dat file on GPUs, a user would first use JtR's bitcoin2john.py and then use hashcat on its output. Conversely, hashcat's 7z2hashcat.pl is the upstream project for JtR's 7z2john.pl, but that isn't as user-visible with 7z2john.pl being in JtR tree. Also importantly, there are differences in maximum supported password lengths. Recent hashcat supports lengths up to 256 for all (non-)hashes through inclusion of deliberately less optimal (slower) implementations along with optimized faster implementations that are more length-limited (with length limits varying by hash type, most painful perhaps being the optimized md5crypt's limit of 15). JtR generally only includes optimized implementations with length limits inherent to those optimizations, with only some exceptions (e.g., on CPU it includes both "md5crypt" with the usual limit of 15 and "md5crypt-long" capable of up to 125, but on GPU it currently only has the optimized "md5crypt-opencl" with the limit of 15). So hashcat fares much better in this respect. Luckily, most other optimization-related length limits are high enough not to pose a problem in practice (e.g., 55 for raw MD5). This issue doesn't apply for most modern (non-)hash types; it only exists for those older and not so well designed password hash types where processing time is significantly affected by password length. There's also not a 100% overlap in which attacks the tools can run, and how. JtR has many candidate password generators on CPU and also uses those to feed candidate passwords to GPUs, with only "mask mode" delegated to GPUs when cracking fast hashes like NTLM. To compensate, JtR is able to stack on-device mask mode on top of other modes, forming hybrid modes where part of the processing occurs on host CPU and part on GPU. This is efficient, but requires that a part of the attack be expressed as a mask. hashcat implements some equivalent (to JtR's) and some different candidate password generators, but has them implemented in device-side code. This makes hashcat more flexible for efficiently running complex attacks against fast hashes on GPU, but makes little difference for attacks on slow (non-)hashes. Also in terms of flexibility, JtR has more candidate password generators (and lets the user write custom ones in a C-like language). For example, its "single crack" mode derives candidate passwords from usernames and other user-specific information and targets them against the specific user's password hash (and other hashes with the same salt), which is efficient when cracking salted hashes. JtR's "Subsets" mode generates strings consisting of few different characters (progressively increasing that number) out of a large set of possible characters. Its "Keyboard" mode (one of the pre-defined custom modes) generates keyboard walk patterns. There are no equivalent modes built into hashcat. On the other hand, hashcat has "Combination" mode, which combines words from multiple wordlists to form candidate passphrases. JtR's closest built-in equivalent is PRINCE mode, kindly contributed by atom of hashcat. So usage of both tools can produce best results. Speaking of which: unfortunately, hashcat tends to greatly under-utilize GPUs when it's fed candidate passwords via stdin, even when the (non-)hash type is slow enough that such under-utilization wouldn't be inherently expected. A workaround is to have "john --stdout" output written out to a file and then read back by hashcat, but that triggers even slower startup. hashcat's startup time feels painfully long for running and adjusting many quick attacks manually, especially on CPU where JtR starts up quickly. This is in part related to hashcat's use of OpenCL even on CPU, but not only that: when run on a new large wordlist, hashcat spends a lot of time "caching" it, whereas JtR can start cracking right away. On CPU, JtR directly (without relying on an OpenCL backend) supports common SIMD instruction sets up to AVX-512 on x86-64. JtR also supports AltiVec on POWER and NEON/ASIMD on ARM/Aarch64. On the other hand, due to NVIDIA's policy of not supporting OpenCL other than on x86-64, JtR is unable to use NVIDIA GPUs on non-x86-64. Recent git hashcat regained such ability through its reintroduction of CUDA support, but I doubt there's a decent OpenCL implementation to use such non-x86 CPUs with hashcat. So if you're on a platform like this and want to use CPUs, use JtR. If you're on an exotic platform like this and want to use NVIDIA GPUs, use hashcat. For example, use JtR on a Raspberry Pi, but use hashcat on a NVIDIA Jetson. You can also use both JtR and hashcat on NVIDIA Jetson to use its CPU and GPU, respectively. Of course, these devices are slow, so this is for casual/hobbyist use. JtR also runs on plenty of other CPU architectures (without SIMD), for which there might not exist OpenCL support. JtR directly supports ZTEX 1.15y FPGA boards (including many at once). Unfortunately, these are now rare. Yet for those who have them, JtR provides very good speeds at bcrypt (higher than hashcat's on high-end GPUs in a per-board comparison) and at descrypt, while also supporting several other common hash types. hashcat could theoretically support FPGAs through OpenCL (and JtR could too), but this doesn't work well in practice (if at all). There are third-party tools that use hashcat and/or JtR. For example, for hashcat there's Hashtopolis: https://github.com/s3inlc/hashtopolis For JtR there's ByePass: https://github.com/webpwnized/byepass If someone wants the functionality provided by these tools, this also determines whether to use hashcat or JtR. There are binary packages of hashcat and/or JtR in various repositories. The user's operating system distribution of choice might have just one of those tools. Besides, it might not provide the most optimal OpenCL backend for licensing reasons. For example, Kali Linux appears to provide pocl instead of Intel's OpenCL, on which hashcat complains: "Not a native Intel OpenCL runtime. Expect massive speed loss." There's no such issue with JtR as packaged in Kali Linux. The above aspects are very far from a complete comparison of the tools. The examples given are just that. There can be many more such examples - for different (non-)hash types, different attacks, different hardware, different OS - where one tool or the other would have advantages. I also started writing a list of what's the same or similar between the tools. It quickly got way too long to include in this same message, so I abandoned it for now. I might try and "complete" and post it later. I'd appreciate any comments, any major differences I might have missed, and especially corrections if I got anything wrong. Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.