john-dev - combine self-test and benchmark

Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <CABtNtWEKiEhZURBRNtTaEgwMsFRLFogdO3722pBmRs401yRiQw@mail.gmail.com>
Date: Thu, 27 Aug 2015 20:32:00 +0800
From: Kai Zhao <loverszhao@...il.com>
To: john-dev@...ts.openwall.com
Subject: combine self-test and benchmark

Hi Alexander,

On Mon, Jun 22, 2015 at 11:36 AM, Solar Designer <solar@...nwall.com> wrote:
>
> The extensive test should mimic actual cracking (testing groups of mixed
> correct and incorrect passwords at once) and perhaps also combine it
> with benchmarking.  Right now, our --test starts with a quick self-test
> and then proceeds with a benchmark, which are separate stages; with an
> extensive self-test that mimics actual cracking, the self-test and
> benchmark should be one and the same stage.

The --test does this:

benchmark_all()
{
        [...]

        benchmark_format();

        if (benchmark_length != -1 && benchmark_length != -1001)
                benchmark_format();

        [...]
}

benchmark_format()
{
        [...]

        fmt_self_test();

        [...]

        // run 1 seconds
        {
                set_key(); // From 0 ~ max - 1
                set_salt();
                crypt_all();
                cmp_all();
        }

        [...]
}

fmt_self_test_full_body()
{
        [...]

        for each test vectors
        {
                prepare();
                valid();
                split();
                binary();
                test_fmt_flags();
                salt();
                set_key();
                crypt_all();
                cmp_all();
                cmp_one();
                get_hash();
                binary_hash();
                cmp_exact();
                get_key();
        }

        [...]
}

1. The self_test() will run two time when run benchmark and benchmark_length
is not -1 or -1001. Should we change it to only one time ?

2. Should we add more functions to the core benchmark procedure ?

benchmark_format()
{
        [...]

        // run 1 seconds
        {
                set_key(); // From 0 ~ max - 1
                set_salt();
                crypt_all();
                cmp_all();
        }

        [...]
}

3. How to combine the self-test and benchmark ? I think there are two
ways:

3.1 Put the benchmark after self-test which is only to move the benchmark
into self_test().

fmt_self_test_full_body()
{
        [...]

        for each test vectors
        {
                prepare();
                valid();
                split();
                binary();
                test_fmt_flags();
                salt();
                set_key();
                crypt_all();
                cmp_all();
                cmp_one();
                get_hash();
                binary_hash();
                cmp_exact();
                get_key();
        }

        // run 1 seconds
        {
                set_key(); // From 0 ~ max - 1
                set_salt();
                crypt_all();
                cmp_all();
        }

        [...]
}

3.2 Remove the benchmark and run self_test() 2~3 seconds to instead
of benchmark(). In this way, there are many functions taken into
benchmarks.

fmt_self_test_full_body()
{
        [...]

        // Each run 2~3 seconds ?
        for each test vectors
        {
                prepare();
                valid();
                split();
                binary();
                test_fmt_flags();
                salt();
                set_key();
                crypt_all();
                cmp_all();
                cmp_one();
                get_hash();
                binary_hash();
                cmp_exact();
                get_key();
        }

        [...]
}


Thanks,

Kai
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.