[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20250610013519.GA319@openwall.com>
Date: Tue, 10 Jun 2025 03:35:19 +0200
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: Problem running john with --rules=oi
On Sat, Jun 07, 2025 at 01:41:37PM -0700, jeff wrote:
> Hearing nothing, I decided to try a newer john.
I'm sorry no one replied, but trying a newer version was a good idea for
other reasons anyway.
> john.exe --fork=34 --format=NT --verbosity=2 --no-log
> --wordlist=\pw-crack\dictionaries\rockyou2021.dic --rules=oi
> \pw-crack\pwn_ntlm_75.7m.rawest
I don't recall if we ever mentioned this before, but I guess that if you
add --keep-guessing you'll be able to use --fork=128 or whatever you
said your actual CPU count was. The reason you had to use lower process
counts is probably primarily that their total memory usage would grow as
they remove the hashes they cracked from memory, which causes
copy-on-write, or in other words unsharing of previously shared memory.
When you let them keep the cracked hashes in memory (and maybe crack
them again), you avoid this unsharing. (At least this is what happens
on Linux. I'm not sure about Cygwin's fork() emulation on Windows.)
--keep-guessing may not have been a good idea for you early on when
cracks were extremely frequent, so .pot file would end up with too many
duplicates. But now that successful cracks are relatively rare, it may
work better (and you can remove the duplicates later with "unique").
> After a few days, I got one more hash, and then the memory usage dropped
> by 50%.
This suggests that some of the child processes terminated, which may be
because they actually completed their portions of work, or for some
other reasons. You may try omitting --no-log and see what's in the log
files for those processes.
Indeed, given that you show progress at 0.26% it's not expected any
would complete their portions so soon. With skipping a lot of rules or
candidates (see below), it is possible that the split in candidates
generated would be very uneven, however this would only result in very
different termination times when targeting a slow hash (or many salts),
not in your case when NT hashes are roughly as quick to compute as words
are to skip.
So it does look like in your case something went wrong.
> I tried to see the status of john running by pressing space and I got no
> response from the prompt where I started it.
This may sometimes be because the rules the main process was going
through at that time end up rejecting all input words. We have many
"reject this word if/unless" conditions within the "oi" rules -
specifically, for would-be overstrikes/inserts beyond the input word
length. Say, a rule trying to change a character at position 14 yet
having to skip millions of shorter input words may appear as
stuck/unresponsive until it's done with that. This is indeed
unfortunate from user experience perspective.
Or it could be for some other reason... The above is just the common,
known-shortcoming not-a-code-bug reason.
Note that the "oi" ruleset starts by targeting lower character positions
and progresses towards higher ones. This means that it will encounter
those large blocks to skip later on.
OTOH, it looks like the wordlist rockyou2021.dic does have some long
lines in it, and in fact you show some on the status line. So such
complete unresponsiveness is unexpected for it, and you're probably
hitting some other issue.
Potentially something related to the ultimate crash on --restore:
> >33 4468g 2:07:01:07 0.26% (ETA: 2027-10-30 03:52) 0.02256g/s 2319Kp/s
> >2319Kc/s 173585GC/s ultimatehunterjumpemr
> > 0 [main] john-avx2-omp 2045
> >fhandler_dev_zero::fixup_mmap_after_fork: requested 0x6FFF77FE0000 !=
> >0x0 mem alloc base 0x0, state 0x10000, size 17547038097408, Win32
> >error 1455
Somehow this lists a size of 17 TB, but this could be just address space
rather than memory, so isn't necessarily a problem.
> > 304 [main] john-avx2-omp 2045
> >K:\pw-crack\john_bleeding_alt\run\john-avx2-omp.exe: *** fatal error
> >in forked process - recreate_mmaps_after_fork_failed
> > 0 [main] john 2012 dofork: child -1 - forked process 39004 died
> >unexpectedly, retry 0, exit code 0x100, errno 11
> >1: fork: Resource temporarily unavailable
This suggests a bug in Cygwin's fork() emulation, or maybe the system
running out of memory. It's surprising it would already at load time
if it didn't for 2 days of running with the same process count before.
A guess is it has something leftover from the previous failed run still
in memory maybe? Can you try this --restore thing after a reboot, or at
least carefully check that you have the usual amount of free memory and
nothing still running in Task Manager?
I also recommend --keep-guessing and moving to Linux. Doing this on
Windows is a stretch (but maybe is part of the fun/challenge for you?)
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
