Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20150504200435.GA2002@openwall.com>
Date: Mon, 4 May 2015 23:04:35 +0300
From: Aleksey Cherepanov <lyosha@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: Generic parsing functions -- prototype

On Mon, May 04, 2015 at 06:33:48PM +0300, Aleksey Cherepanov wrote:
> On Mon, Mar 30, 2015 at 02:29:46AM +0300, Alexander Cherepanov wrote:
> > - hex. Do we need variants for lower- and upper-case?
> 
> From formats.h:
> /* Splits a ciphertext into several pieces and returns the piece with given
>  * index, starting from 0 (will usually return the ciphertext unchanged).
>  * For hex-encoded hashes which are compared by the target system/application
>  * irrespective of the case of characters (upper/lower/mixed) used in their
>  * encoding, split() must unify the case (e.g., convert to all-lowercase)
>  * and FMT_SPLIT_UNIFIES_CASE must be set. */
> 	char *(*split)(char *ciphertext, int index, struct fmt_main *self);
> 
> For example, raw-sha512 reads hex in any case and makes lower case in
> split().
> 
> Flags in fmt_main of raw-sha512:
> 		FMT_CASE | FMT_8_BIT | FMT_OMP | FMT_SPLIT_UNIFIES_CASE,

For raw-sha512, there may be just 64 chars in hex or tag and then 64
chars in hex. I'd like to see fixed length for hex (currently only max
length may be specified). I'd like to see a function to be used in
split() that makes canonical hash: adds the tag and lowercases the hex
part. (binary() is called after split().)

I tried to use the lib. So far I have:

#define HASH_FORMAT "%*h"
#define HASH_FORMAT_TAGGED FORMAT_TAG HASH_FORMAT

static int valid(char *ciphertext, struct fmt_main *self)
{
    return proc_valid(ciphertext, HASH_FORMAT, BINARY_SIZE)
        || proc_valid(ciphertext, HASH_FORMAT_TAGGED, BINARY_SIZE);
}

Original split()

static void *binary(char *ciphertext)
{
    static unsigned char buf[BINARY_SIZE];
    size_t len;
    proc_extract(ciphertext, HASH_FORMAT_TAGGED, &len, buf);
    return buf;
}

(    proc_extract(ciphertext, HASH_FORMAT_TAGGED, IGNORE_NUM, buf);
gave me segfault there.)

Most probably static buffer has to be replaced with static pointer to
reduce size of john's binary:
    static unsigned char *buf;
    if (!buf)
        buf = mem_alloc_tiny(BINARY_SIZE, MEM_ALIGN_WORD);


The code works for good bare hashes. But it will work wrong for
shorter hashes. It fails self tests because $SHA512$ without hex
part is tried:
FAILED (promiscuous valid ($SHA512$))

Interestingly format tag is not part of fmt_main, the tag is extracted
from self tests matching $ (so the test will not be applied if the tag
is not in $).

It is possible to write valid() using the lib as is but it would be
ugly. I think %0-*h or %1-*h may be used to specify variable length,
while %10h should specify fixed length like %10-10h but with 1 value
to be specified in case of *. Though %10h and %10-10h may be different
in another way too: it is not needed to return the length with %10h.

Thanks!

-- 
Regards,
Aleksey Cherepanov

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.