Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20110817113028.GA27401@openwall.com>
Date: Wed, 17 Aug 2011 15:30:28 +0400
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: 45% wordlist boost just waiting to happen

On Mon, Aug 15, 2011 at 09:35:44PM +0200, magnum wrote:
> These lines in wordlist.c always nagged me:
> 
> 
>   #if 0 && ARCH_ALLOWS_UNALIGNED
>   /* XXX: somehow this breaks things - why? */
>           line = words[nCurLine++];
>   #else
>           strcpy(line, words[nCurLine++]);
>   #endif

As far as I recall, JimF's original patch adding in-memory wordlists
only had the strcpy() here.  When merging the patch, I noticed that this
was inefficient, so I added the #if ARCH_ALLOWS_UNALIGNED and pointer
assignment.  However, somehow this did not work right.  I did not
investigate, and simply added the "0 &&" then.  That's all.

What I need to do is reimplement in-memory wordlists myself, in the
main JtR tree.

BTW, there's also a bug that I happened to introduce in 1.7.4 when I
moved the dupe detection from wordlist.c into rules.c (for the case when
rules are enabled).  The problem is that when making this change I
forgot about external filter() possibly modifying the word.  So the dupe
detection happens to compare a pre-filter() new word with post-filter()
previous word.  I might fix this at the same time, or separately...
although it may be tricky to fix without performance impact.

> Today I finally tried removing that "0 && " road block and this made for 
> a stunning 45% boost running a wordlist + rules against LM.

That's impressive.  Perhaps when you apply the -fast-des-key-setup-3
patch, the boost for LM may be even higher.

> I see no problems.

Well, you do now (I am referring to your follow-up posting).  There's
probably a bug somewhere else (likely in the in-memory wordlists code as
well) that this change triggers.

Alexander

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.