Date: Tue, 06 Jun 2006 22:11:44 +1000 From: David Luyer <david@...er.net> To: <john-users@...ts.openwall.com> Subject: Re: to MPI or not MPI john? On 6/6/06 9:20 PM, "Randy B" <aoz.syn@...il.com> wrote: > I know a lot of parallelization people want to see an approach that > allows loose clusters to work on a given problem set, but I for one am > interested simply to use my SMP machines more efficiently. I think it > would move us along much more quickly (and scratch my itch sooner... > ;-) ) if focus was initially placed on solving the parallelization > problem, then on distribution to multiple nodes. If I *really* want > to run a non cluster-aware program on multiple nodes of a cluster, > I'll wrap it in something like Mosix. I think I've mentioned my quick and dirty approach to distributing john before, both on SMP and on multiple machines; ignore the early passes because they all happen within a relatively short period, and focus on incremental, then just distribute individual incremental passes to individual machines, think about how large-scale distributed efforts (SETI@...e etc) work, and map that to john; - hand out work block to client: - client requests work block - server answers: you have incremental pass #5 to work on - client works on incremental pass #5 - wait for completion of work block - hand out new work block - repeat The advantage is that you don't have to worry about splitting the work evenly, servers falling behind (if they have unequal capability or unequal hash distribution), wasted CPU in generating keys which won't be checked, etc. I've only done this in dodgy ways with manual editing on the status files and merging of john.pot files etc. But it could easily be automated as part of john itself. The issues to think about are: - marking a work block as "not done" if a server dies while working on it, losing its state file - automatically distributing updates to passwords cracked (can't assume NFS distribution of john.pot, some people may wish to utilize CPU resources across multiple security domains) - having the one server hand out tasks on hash types/workloads? (it is not uncommon to have a mix of 3DES and MD5 and want to find plaintexts for both) David.
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.