Subject: Re: [OMPI users] Spawn and OpenFabrics
From: Allen Barnett (allen_at_[hidden])
Date: 2009-06-07 21:19:19
OK. I appreciate the suggestion and will definitely try it out.
Thanks,
Allen
On Fri, 2009-06-05 at 10:14 -0400, Jeff Squyres wrote:
> On Jun 2, 2009, at 3:26 PM, Allen Barnett wrote:
> > I
> > guess what I'm asking is if I will have to make my partitioner an
> > OpenMPI program as well?
> >
>
>
> If you use MPI_COMM_SPAWN with the 1.2 series, yes.
>
> Another less attractive but functional solution would be to do what I
> did for the new command notifier due in the OMPI v1.5 series
> ("notifier" = subsystem to notify external agents when OMPI detects
> something wrong, like write to the syslog, send an email, write to a
> sysadmin mysql db, etc., "command" = plugin that simply forks and runs
> whatever command you want). During MPI_INIT, the fork notifier pre-
> forks a dummy process. This dummy process then waits for commands via
> a pipe. When the parent (MPI process itself) wants to fork a child,
> it sends the argv to exec down the pipe and has the child process
> actually do the fork and exec.
>
> Proxying all the fork requests through a secondary process like this
> avoids all the problems with registered memory in the child process.
> This is icky, but it is an unfortunately necessity for OS-bypass/
> registration-based networks like OpenFabrics.
>
> In your case, you'd want to pre-fork before calling MPI_INIT. But the
> rest of the technique is pretty much the same.
>
> Have a look at the code in this tree if it helps:
>
> https://svn.open-mpi.org/trac/ompi/browser/trunk/orte/mca/notifier/command