Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Tue, 16 Jun 2009 12:15:20 -0600
From: RB <>
Subject: Re: MPI John (was: Re: GI John)

On Tue, Jun 16, 2009 at 10:50, Minga Minga <> wrote:
> The main questions I have are :
> Is MPI+incremental mode  faster than --external:Parallel ?
> If so, by what percentage? Benefits ? Drawbacks?

Single-threaded JtR + MPI is slower than JtR by itself; I don't have
the numbers here with me, but IIRC the difference is marginal (if
distinct).  Hence, the MPI implementation is likely slower than
--external:Parallel.  What the MPI patch does add is scalability - I
personally am willing to sacrifice several percentage points in
efficiency for the ability to trivially spread processing over N cores
in a large, potentially heterogeneous cluster without having to
distribute custom per-node configurations.  I've seen indications that
it doesn't work well for very large clusters, but the largest I've
personally done was 28 cores.

> Can I trust MPI to not duplicate effort?

I believe so.  The modifications to src/inc.c basically cause each MPI
thread to leapfrog through the incremental steps.  When looking at the
log file, a single-threaded run goes through each of the "Trying
length X, fixed @Y, character count Z" steps in-order.  With the MPI
patch, core0 will run "1..N step C", core1 will do "2..N step C", and
so on.  It's non-optimal and can leave holes in an unbalanced cluster,
but it is what it is.

> Am I wasting CPU cycles using --external:Parallel for large "incremental"
> runs?

Doubtful, especially if --external:Parallel fulfills your needs.  The
Lim MPI patch is far less granular on its split than
--external:Parallel, but I'm doubtful the additional per-word modulus
adds up to a significant reduction in efficiency.  YMMV, and I'm sure
someone else here has already run the math.

To unsubscribe, e-mail and reply
to the automated confirmation request that will be sent to you.

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.