Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Tue, 21 Apr 2015 17:18:22 +0300
From: Aleksey Cherepanov <lyosha@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Johnny: core/jumbo differences

I've tried to write down exhaustive list of differences between john
core and john jumbo. I don't know everything, so feel free to comment.

This list does not cover usage of the differences in Johnny. I'll make
a follow up with ideas.

Thanks!

Regards,
Aleksey Cherepanov



* Common CLI options without differences

--incremental[=MODE]       "incremental" mode [using section MODE]
--external=MODE            external mode or word filter


--stdout[=LENGTH]          just output candidate passwords [cut at LENGTH]
--restore[=NAME]           restore an interrupted session [called NAME]
--session=NAME             give a new session the NAME
--status[=NAME]            print status of a session [called NAME]


There is a point about session names related to --fork/--node: there
are multiple session files with number of node in names.

$ ./core-1.8.0/run/john  ~/d/disk/contest/hashes-canon/01-base.md5crypt.pw
$ ls core-1.8.0/run/*rec
core-1.8.0/run/john.rec
$ ./core-1.8.0/run/john --fork=3  ~/d/disk/contest/hashes-canon/01-base.md5crypt.pw
$ ls core-1.8.0/run/*rec
core-1.8.0/run/john.2.rec  core-1.8.0/run/john.3.rec  core-1.8.0/run/john.rec
$ rm core-1.8.0/run/*rec
$ ./core-1.8.0/run/john --fork=3 --node=5-7/16 ~/d/disk/contest/hashes-canon/01-base.md5crypt.pw
$ ls core-1.8.0/run/*rec
core-1.8.0/run/john.6.rec
core-1.8.0/run/john.7.rec
core-1.8.0/run/john.rec


--make-charset=FILE        make a charset, FILE will be overwritten
--make-charset=FILE       make a charset file. It will be overwritten
(core, then jumbo; only descriptions are different)


--test[=TIME]              run tests and benchmarks for TIME seconds each

--users=[-]LOGIN|UID[,..]  [do not] load this (these) user(s) only
--groups=[-]GID[,..]       load users [not] of this (these) group(s) only
--shells=[-]SHELL[,..]     load users with[out] this (these) shell(s) only

--save-memory=LEVEL        enable memory saving, at LEVEL 1..3

--node=MIN[-MAX]/TOTAL     this node's number range out of TOTAL count
--fork=N                   fork N processes


* Common CLI Options

The format:
core option
jumbo option
comments

----------------------------------------

--single                   "single crack" mode

--single[=SECTION]        "single crack" mode

It is possible to choose Rules section to run with single mode.
[List.Rules:Single] is the default in both versions (so --single is
equivalent to --single=Single; though parameters are not case
sensitive).

----------------------------------------

--wordlist=FILE --stdin    wordlist mode, read words from FILE or stdin

--wordlist[=FILE] --stdin wordlist mode, read words from FILE or stdin
                  --pipe  like --stdin, but bulk reads, and allows rules

$ ./core-1.8.0/run/john --wordlist ~/d/t.pw
Option requires a parameter: "--wordlist"

The default wordlist is used by default in jumbo:
[Options]
# Wordlist file name, to be used in batch mode
Wordlist = $JOHN/password.lst

Also jumbo has --stdin and --pipe options to read candidates from
standard input.

----------------------------------------

--rules                    enable word mangling rules for wordlist mode

--rules[=SECTION]         enable word mangling rules for wordlist modes

$ ./core-1.8.0/run/john --rules=NT --wordlist=~/d/t.pw ~/d/t.pw
Extra parameter for option: "--rules=NT"

Core does not allow rules section to be specified. Both versions
default to [List.Rules:Wordlist]

----------------------------------------

--show                     show cracked passwords

--show[=LEFT]             show cracked passwords [if =LEFT, then uncracked]

In jumbo --show can print remaining hashes but it drops dupes (when
there is 1 canonical hash for 2 users and the hash is in different
forms and/or there are different gecos).

----------------------------------------

--salts=[-]N               load salts with[out] at least N passwords only

--salts=[-]COUNT[:MAX]    load salts with[out] COUNT [to MAX] hashes

----------------------------------------

--format=NAME              force hash type NAME: descrypt/bsdicrypt/md5crypt/
                           bcrypt/LM/AFS/tripcode/dummy/crypt

--format=NAME             force hash of type NAME. The supported formats can
                          be seen with --list=formats and --list=subformats

Format list is much larger in jumbo, also it is possible to specify
hash type for generic crypt through --subformat option.


* Jumbo-only CLI options

--loopback[=FILE]         like --wordlist, but fetch words from a .pot file
--dupe-suppression        suppress all dupes in wordlist (and force preload)
--prince[=FILE]           PRINCE mode, read words from FILE
--encoding=NAME           input encoding (eg. UTF-8, ISO-8859-1). See also
                          doc/ENCODING and --list=hidden-options.
--mask=MASK               mask mode using MASK
--markov[=OPTIONS]        "Markov" mode (see doc/MARKOV)
--pot=NAME                pot file to use
--list=WHAT               list capabilities, see --list=help or doc/OPTIONS
--help                    print usage summary, just like running the command
                          without any parameters
--config=FILE             use FILE instead of john.conf or john.ini
--mem-file-size=SIZE      size threshold for wordlist preload (default 5 MB)
--format=CLASS            valid classes: dynamic, cpu
--subformat=FORMAT        pick a benchmark format for --format=crypt
--mkpc=N                  request a lower max. keys per crypt
--min-length=N            request a minimum candidate length
--max-length=N            request a maximum candidate length
--costs=[-]C[:M][,...]    load salts with[out] cost value Cn [to Mn] for
                          tunable cost parameters, see doc/OPTIONS
                          (comma separated list of values/ranges per param.)
--field-separator-char=C  use 'C' instead of the ':' in input and pot files
--fix-state-delay=N       performance tweak, see doc/OPTIONS
--nolog                   disables creation and writing to john.log file
--log-stderr              log to screen instead of file
--bare-always-valid=C     if C is 'Y' or 'y', then the dynamic format will
                          always treat bare hashes as valid
--progress-every=N        emit a status line every N seconds
--crack-status            emit a status line whenever a password is cracked
--keep-guessing           try more candidates for cracked hashes (ie. search
                          for plaintext collisions)
--max-run-time=N          gracefully exit after this many seconds
--regen-lost-salts=N      regenerate lost salts (see doc/OPTIONS)
--mkv-stats=FILE          "Markov" stats file (see doc/MARKOV)
--reject-printable        reject printable binaries
--verbosity=N             change verbosity (1-5, default 3)
--skip-self-tests         skip self tests
--stress-test[=TIME]      loop self tests forever
--input-encoding=NAME     input encoding (alias for --encoding)
--internal-encoding=NAME  encoding used in rules/masks (see doc/ENCODING)
--target-encoding=NAME    output encoding (used by format, see doc/ENCODING)

PRINCE mode options:
--prince-loopback[=FILE]  fetch words from a .pot file
--prince-elem-cnt-min=N   minimum number of elements per chain (1)
--prince-elem-cnt-max=N   maximum number of elements per chain (8)
--prince-skip=N           initial skip
--prince-limit=N          limit number of candidates generated
--prince-wl-dist-len      calculate length distribution from wordlist
                          instead of using built-in table
--prince-wl-max=N         load only N words from input wordlist
--prince-case-permute     permute case of first letter
--prince-mmap             memory-map infile (not available when permuting case)
--prince-keyspace         just show total keyspace that would be produced
                          (disregarding skip and limit)

Also there should be options for rexgen and other things not compiled
in my john:
OpenMPI support (default disabled) .......... no
OpenMP support .............................. no
OpenCL support .............................. no
CUDA support ................................ no
Experimental code ........................... no
Rexgen (extra cracking mode) ................ no
Memdbg memory debugging settings ............ disabled
AddressSanitizer ("ASan") ................... disabled


* Config file

In Jumbo config files may include other files. Core john has only one
file.

John built from sources searches config file (and other files) in run/
directory, john installed from package most usually looks into ~/.john
and /etc/john . While distros usually use john core, there distros
with jumbo. Also this behaviour may depend on distro.

With jumbo, you can get path to the folder with --list=build-info option:
$ jumbo/run/john --list=build-info
[...]
$JOHN is jumbo/run/
[...]


* Utilities and tools

In core
john - john
mailer - "script to send mail to all users whose passwords got cracked."
makechr - utility to make .chr files (incremental mode) from current .pot
relbench - statistics for benchmarks
unafs
unique - filter to remove dupes in wordlist saving order
unshadow

unafs and unshadow may fall into *2john category.


BTW Jumbo has bash auto-completions.


In jumbo there are various utilities:

*2john converters (including symlinks):

These tools usually read file(s) and print hashes to stdout in a form
john understands them. Some utilities perform quite non-trivial
parsing using pulled-in third party libraries.

1password2john.py
7z2john.py
aix2john.pl
aix2john.py
androidfde2john.py
apex2john.py
bitcoin2john.py
blockchain2john.py
cisco2john.pl
cracf2john.py
dmg2john
dmg2john.py
ecryptfs2john.py
efs2john.py
encfs2john.py
gpg2john
hccap2john
htdigest2john.py
ikescan2john.py
ios7tojohn.pl
kdcdump2john.py
keepass2john
keychain2john
keychain2john.py
keyring2john
keyring2john.py
keystore2john
keystore2john.py
known_hosts2john.py
kwallet2john
kwallet2john.py
lastpass2john.py
ldif2john.pl
lion2john-alt.pl
lion2john.pl
lotus2john.py
luks2john
mcafee_epo2john.py
ml2john.py
mozilla2john.py
odf2john.py
office2john.py
openbsd_softraid2john.py
openssl2john.py
pcap2john.py
pdf2john.py
pfx2john
putty2john
pwsafe2john
racf2john
radius2john.pl
rar2john
raw2dyna
rexgen2rules.pl
sap2john.pl
sipdump2john.py
ssh2john
ssh2sshng.py
sshng2john.py
strip2john.py
sxc2john.py
truecrypt_volume2john
uaf2john
vncpcap2john
wpapcap2john
zip2john

undrop


There are more tools:

base64conv - raw/hex/base64(mime/crypt/cryptBS) converter
genincstats.rb - make statistics from .inc file
leet.pl - rules generator for leet speak
netscreen.py - netscreen hasher
hextoraw.pl - unhex for pass_gen.pl
pass_gen.pl - dynamic formats hasher
sha-dump.pl - ldap dumper
sha-test.pl - ldap sha1 hasher
calc_stat - some statistics for wordlist

benchmark-unify
# John the Ripper benchmark output conversion tool, revision 1

netntlm.pl
#   The purpose of this script is to aid with cracking a LM/NTLM
#   challenge/response set, when part of the password is known. It
#   was written with John's NetLM/NetNTLM formats and "halflmchall"
#   Rainbow Tables in mind.

unrule.pl
# Extract basewords from list of plains. Based on an embryo from epixoip.
# ./unrule.pl < rockyou.lst > basewords.lst

(I don't know at all)
genmkvpwd
mkvcalcproba
SIPdump
tgtsnarf
cprepair

Powered by blists - more mailing lists

Your e-mail address:

Powered by Openwall GNU/*/Linux - Powered by OpenVZ