john-dev - Re: testing every index (Re: more robustness)

Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sun, 23 Aug 2015 00:00:19 +0800
From: Kai Zhao <loverszhao@...il.com>
To: john-dev@...ts.openwall.com
Subject: Re: testing every index (Re: more robustness)

Hi Alexander,

On Sat, Aug 22, 2015 at 9:34 AM, Solar Designer <solar@...nwall.com> wrote:
>
> There's no such thing as "Current max max_keys_per_crypt".  I guess you
> meant for the current version of JtR, but no, that's not how it works.
> max_keys_per_crypt is adjusted dynamically by many formats' init(), and
> how it's adjusted depends on things such as OpenMP thread count and
> OpenCL GWS tuning.  Your current 33554432, while extremely high, might
> in fact not be enough for current JtR on some hardware.
>
> Also, you used a static array for this.  This grows the .bss
> unnecessarily.  Please use dynamic memory allocation for the actual
> max_keys_per_crypt instead.  If you don't want to allocate/free for each
> format, you may use realloc() to a higher size whenever a next format
> has a higher max_keys_per_crypt than the previously seen maximum (in the
> formats tested up to that point).
>
> For the correct password, you're still setting just one password for all
> indices.  You should instead use all passwords from the test vectors,
> then compare each computed hash against the one expected for the test
> vector that you used for that index.
>
> The above means that your Boolean correct_key[] is better changed to
> array of pointers (pointing to the test vector hashes, or NULL if not
> correct) or integers (storing test vector numbers, or e.g. -1 if not
> correct).
>
> Please make the above enhancements.

A new patch for testing every index:

https://github.com/loverszhaokai/JohnTheRipper/commit/7fcc07ef25464907105c266fa2b9caae0b598813

There are 3 formats have fatal errors: "cmp_exact() unexpected success"

1. AFS

static struct fmt_tests tests[] = {
0        {"$K4$e35e9294ecef926d,0123", "U*U*U*U*"},
1        {"$K4$64c7c2aedccd70d6,0123456789", "U*U***U*"},
2        {"$K4$d9e985b36268f168,01234567", "U*U***U"},
3        {"$K4$b9615786dfb53297,longcellname", "longpassword"},
4        {"$K4$a8dc8aeaa2c48a97,", ""},
5        {"$K4$dfda85c7619183a2,XXXXXXXX", "XXXXXXXX"},
6        {"$K4$e3e59de6f1d5eaf4,cell", "password355"},
7        {"$K4$b02cc24aefbc865b,", "thisisaverylongpassword"},
          {NULL}
};

When testing the 4th test vector, "" is correct password, but
the cmp_exact()  returns 1 when incorrect password is
"XXXXXXXX" which is the 5th plaintext.

Both "" and "XXXXXXXX" are the correct passwords ?

2. bfegg

static struct fmt_tests tests[] = {
0    {"+9F93o1OxwgK1", "123456"},
1    {"+C/.8o.Wuph9.", "qwerty"},
2    {"+EEHgy/MBLDd0", "walkman"},
3    {"+vPBrs07OTXE/", "tesztuser"},
4    {"+zIvO/1nDsd9.", "654321"},
5    {"+V6ZOx0rVGWT0", "1"},
6    {"+V6ZOx0rVGWT0", "11"},
7    {"+Obytd.zXYjH/",
"abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"},
    {NULL}
};

The ciphertext of the 5th and 6th are the same, so when we test
the 5th test vector, the cmp_exact() will return 1 when the incorrect
password is "11". This leads "cmp_exact() unexpected success".

3. mysql is the same with bfegg

static struct fmt_tests tests[] = {
        [...]
        {"1b38cd9c2f809809", "hacktivity2008"},
        {"1b38cd9c2f809809", "hacktivity 2008"},
        [...]
        {NULL}
};

> Meanwhile, have these tests uncovered any bugs?

Currently, I only found the problems with cq, vnc and keyring.


Thanks,

Kai
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.