Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Thu, 12 Apr 2012 11:46:26 -0500
From: "jfoug" <jfoug@....net>
To: <john-dev@...ts.openwall.com>
Subject: RE: PDF format

I was going to comment off-list, to offer to help out some on this.
However, I think a full on-list explanation is not off-topic, especially
since there are likely new people trying to come up to speed, on the lower
level how-to on john.

>This is now done and committed to magnum-jumbo. However "many salts"
>case is still slower than "one salt" case. Is this due to initPDFCrack
>function being called from set_salt?

It appears very likely, this also requires pre-computation, so that it is
ONLY done one time, at program start.  This requires memory.

Thus, you probably want to keep this initPDFCrack, passing in the pointer to
the salt structure being built.  This 'salt' structure would have pointers
for all data created within the initPDFCrack() or any functions it calls.
It looks like a lot of allocations, memcpy's, byte twiddling, etc.  Again,
ALL of this work is being done each set_salt, when we could eliminate this
and do it only once in get_salt (at program load).

However, you still need to 'put' this pre-computed data into the pdfcrack
static pointers (or int sizes, etc).  So there will probably need to be a
loadPDFCrack() function, which you pass in the salt pointer, and this
function loads all the static data, but does not have to compute it again
and again (hopefully everything is just pointers, or ints).


Thus, this will cut the set_salt to:

static void set_salt(void *salt)
{
	salt_struct = (struct custom_salt *)salt;
	if (!loadPDFCrack(salt_struct)) {  /* FAST copying of pre-computed
data using pointers */
		cleanPDFCrack();
		fprintf(stderr, "Wrong userpassword, '%s'\n",
salt_struct->userpassword);
		exit(-1);
	}
}

Where the 'current' set_salt is:

static void set_salt(void *salt)
{
	salt_struct = (struct custom_salt *)salt;
	/* initPDFCrack does a ton of work, each time a new salt is selected
*/
	if (!initPDFCrack(&salt_struct->e, salt_struct->userpassword,
salt_struct->e.work_with_user)) {
		cleanPDFCrack();
		fprintf(stderr, "Wrong userpassword, '%s'\n",
salt_struct->userpassword);
		exit(-1);
	}
}


You have removed much of the long loader code, from set_salt, and put it in
get_salt.  However, there is a lot more of this loader code 'hidden' in this
initPDFCrack function.  This is some of the 'fun' of trying to do C++'ish
class hiding of data, within C (multiple files, vs classes, and static data,
vs private class member data).


But to do any of these, you will have to change your 'custom_salt' structure
(AND have this structure layout known by more than one source file).

Something like this might be needed (note, this will likely be FAR from all
that's needed, but should give you the hints needed)

In pdf_fmt.h file:

struct custom_salt {
	struct EncData e;
	unsigned char *userpassword;
	// below data will be loaded into pdfcrack.c, in a new
loadPDFCrack() function call.  But we build this data one time only.
	unsigned int ekwlen;
	uint8_t * encKeyWorkSpace;
	uint8_t *password_user;
	uint8_t *rev3TestKey;
	// int binitPDFCrack_called = 0;  // likely NOT part of the salt,
but set by calling loadPDFCrack.
	unsigned char *currPW;
	unsigned int currPWLen;
	bool knownPassword;
	bool workWithUser;
} *salt_struct;

In pdf_fmt.c
static struct custom_salt *salt_struct;

Then, within get_salt, a call to a 'modified' initPDFCrack is calls, passing
in the custom_salt variable.  All work is done, allocations, etc, on the
salt data.

Then, within set_salt, we simply call the 'new' loadPDFCrack() function,
passing in our salt, and that function quickly sets up the data, and
prepares it to properly handle the runCrack() call done by the formats
crypt_all function call.  

ALSO NOTE NOTE NOTE, cleanPDFCrack function would NOT delete anything
(likely that function would simply go away, or be reduced to properly
initializing data to 'default' values, such as setting pointers to null).
The initPDFCrack() code is done one time, and only one time for a salt, and
that prepared data will live for the entire run of john, and be used over
and over again (for each salt.)

NOTE2, I do not 'know' the format.  It may be that a bit of cleanup may be
required, more than simply copying the password into the encWorkSpace.  We
may have to do a tiny bit of cleanup, of the last password.  

But once done, all salt 'building' is done 1 time, upfront.  The set_salt()
simply uses this pre-built data.  This should increase the multiple salts
speed greatly, because all of the salt processing time will be eliminated
from the inner-loop, and THIS processing (in the inner loop), is redundant.

Jim.

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.