Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Thu, 8 Sep 2016 20:03:53 +0200
From: magnum <>
Subject: Re: possible memory leak on FreeBSD?

On 2016-09-08 08:42, wrote:
> I'm running JtR (JohnTheRipper-bleeding-jumbo 20160728) on FreeBSD 10.1-RELEASE, and I'm experiencing some nasty memory problem with some settings.
> I'm cracking huge passwords dump (10s of millions records), and my current pot file is about 4.3 GB. The server has 16 GB ram (but also runs other softwares).
> For example, --incremental will apparently very slowly consume memory on this server. I can't make really sure about this, but I can see the consumed swap size slowly increase overnight. Under normal usage, this server never swap a single bit.
> It becomes blatant when I use --fork=4 with --incremental: the memory is exhausted in about 10-30 minutes and swap piles up. If I don't kill john, the box ends up crashing (swap exhaustion on ZFS is not good). Oddly, top output does not show a real increase in john's memory usage while free memory on host is depleting.
> Same goes with --loopback --fork=4, even with a smaller pot file.
> Other attack modes like --wordlist are OK.
> I know nothing about debugging and memory leak tracking. Any clue appreciated.

What format are you seeing this with? We aren't aware of any memory 
leaks right now, but maybe it's some fairly recent bug you are seeing 
(if the problem is with JtR at all). We have a very good way to diagnose 
it though, the "memdbg" debug option to ./configure.


$ ./configure --enable-memdbg && make -s clean && make -sj4

Using a memdbg-build and when you think memory has leaked, you can just 
'q'uit JtR and you'll get diagnostics. Here's example output (after I 
planted a bug in the NT format):

MEMDBG: allocation information (display):
    current normal alloc mem (leaks)192  max normal mem allocated: 143658344
    current 'tiny' alloc mem (leaks)0  max  tiny  mem allocated: 4615809

Index : alloc# :   Size : File(Line)  [first 20 bytes, or size of bytes]
0     : 158    :    192 : nt2_fmt_plug.c(220)  ..............2h=.+.
At Program Exit
MemDbg_Validate level 0 checking Passed

As you see, it points to an alloc that were never freed later.

But PLEASE NOTE that I just now, 10 minutes ago, committed a bug fix for 
memdbg: Prior to that fix, you'd only get diagnostics if the session 
ended because all hashes were cracked OR the mode exhausted. After the 
fix, you'll also get diagnostic if aborting (ctrl-c or 'q') or if 
--max-run-time=SECS was used and timer ran out. Thank you for making me 
discover that bug ;-)


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.