Date: Fri, 29 Jan 2016 22:25:44 -0500 From: japhar81 <japhar81@...il.com> To: john-users@...ts.openwall.com Subject: Re: MPI with Spot Instances? They basically just disappear, whatever they were in the middle of -- i guess my question is, will the resume re-run whatever jobs those nodes were in the middle of and didn't report back? And if one of them happens to have hit a match, will that get saved somehow too? On Fri, Jan 29, 2016 at 10:20 PM, magnum <john.magnum@...hmail.com> wrote: > On 2016-01-29 18:12, japhar81 wrote: > >> Ok, so corollary question -- does the session stuff work with MPI? i.e. >> lets say I start the spot instances externally, and mpiexec jtr with some >> flavor of --session (on a box that wont die). If those nodes die >> mid-process, will that be recorded in the session to enable a resume later >> when I spin new nodes and start mpiexec again? >> > > Sure (as far as I can imagine how spot instances work). Session file > integrity is very well tested. > > magnum > > > On Wed, Jan 27, 2016 at 4:03 PM, magnum <john.magnum@...hmail.com> wrote: >> >> On 2016-01-27 17:25, japhar81 wrote: >>> >>> I've been playing around with MPI clustering JtR for a while, and I've >>>> managed to get it running smoothly on static nodes. What I'd like to do >>>> next is create an auto-scaling group in AWS, using spot instances. What >>>> this basically means is nodes will come and go, with their hostnames/IPs >>>> changing at random.. I can not figure out how I would run JtR in that >>>> instance -- since it requires a node list in a file on startup to >>>> mpirun. >>>> >>>> If it matters, I'm looking to do a brute-force using the ASCII mode. Has >>>> anyone found a way to do a dynamic cluster that adds/removes nodes at >>>> random? Is this even possible? >>>> >>>> >>> I'm not aware of any existing work that would do this. A solution using >>> JtR as-is, but with some yet-to-be-implemented master issuing jobs, could >>> involve looking at the existing "-node=x/y" as describing "pieces" >>> instead >>> of "nodes". So instead of saying -node=2/8 as in "you are node 2 of 8" >>> you'd say -node=4321/100000 as in "do piece 4321 of 100000". Then you'd >>> submit pieces to active nodes. Obviously you'd have to handle dying nodes >>> that never reported back their given piece, and re-issue those pieces. >>> >>> magnum >>> >>> >>> >> >
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.