[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 16 Mar 2015 17:58:05 +0100
From: magnum <john.magnum@...hmail.com>
To: john-dev@...ts.openwall.com
Subject: Re: [GSoC] building JtR for MIC
On 2015-03-15 19:14, Solar Designer wrote:
> On Sun, Mar 15, 2015 at 01:12:48PM +0800, Lei Zhang wrote:
>> It got killed again when running smchapv2-naive. I guess it ran out of memory again?
>
> Perhaps. Can you try running the mschapv2-naive self-test on its own,
> not along with the rest? Something like:
>
> ./john --test --format=mschapv2-naive
>
> If it runs fine, then the problem was likely caused by memory allocation
> by the formats that were tested previously. Some of them intentionally
> allocate memory such that it can't be freed. This lowers overhead for
> when john is run for cracking, with just one specific format, but it
> results in increased total memory usage for a many-formats self-test.
> We should only be making such non-freed allocations for things that are
> small. For larger allocations, formats should free the memory in their
> done() methods. Perhaps some formats allocate "too much" memory (at
> least when the number of threads is this large) and don't free it - and
> perhaps you should identify them and change them to free the memory.
This has been on my to-do list for some time. The "worst" formats use an
OMP_SCALE of eg. 128K. Even at just 16K and with a PLAINTEXT_LENGTH of
125, the key buffer alone is nearly half a gigabyte for 240 threads.
I just committed fixes for a good number of the worst offenders.
Hopefully this will fix the problem already but we should fix some more
later. And Lei, if you see a particular format still causing troubles,
feel free to fix that and make a PR.
Basically the below is what's involved for each format:
commit db1f7e7d6a5b0c75baf878f810f6da1bbb0dc79b
Author: magnum <john.magnum@...hmail.com>
Date: Mon Mar 16 15:51:13 2015 +0100
Use malloc/free for buffers; Yet another bunch of formats.
diff --git a/src/sapH_fmt_plug.c b/src/sapH_fmt_plug.c
index a260cbc..f76859a 100644
--- a/src/sapH_fmt_plug.c
+++ b/src/sapH_fmt_plug.c
@@ -154,8 +154,16 @@ static void init(struct fmt_main *self)
omp_t *= OMP_SCALE;
self->params.max_keys_per_crypt *= omp_t;
#endif
- saved_plain = mem_calloc_tiny(sizeof(*saved_plain) *
self->params.max_keys_per_crypt, MEM_ALIGN_NONE);
- crypt_key = mem_calloc_tiny(sizeof(*crypt_key) *
self->params.max_keys_per_crypt, MEM_ALIGN_WORD);
+ saved_plain = mem_calloc(self->params.max_keys_per_crypt,
+ sizeof(*saved_plain));
+ crypt_key = mem_calloc(self->params.max_keys_per_crypt,
+ sizeof(*crypt_key));
+}
+
+static void done()
+{
+ MEM_FREE(crypt_key);
+ MEM_FREE(saved_plain);
}
static int valid(char *ciphertext, struct fmt_main *self)
@@ -658,7 +666,7 @@ struct fmt_main fmt_sapH = {
tests
}, {
init,
- fmt_default_done,
+ done,
fmt_default_reset,
fmt_default_prepare,
valid,
magnum
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
