From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2007-05-18 17:17:31
Keep in mind that there are two kinds of TCP traffic that OMPI uses:
- "OOB" (out of band, meaning non-MPI): startup protocols,
communication with mpirun, etc. This is probably not interesting to
you.
- MPI: the back-end to MPI_SEND and friends.
What I have done is get 2 nodes on my cluster that are otherwise
unrelated meaning that there won't be any other traffic between them
besides MPI (e.g., one is not a file server for the other, etc.): A
and B. Run the application like this:
a$ mpirun -np 2 --host a,b --mca btl tcp,self a.out
Then in another shell, run tcpdump and capture any TCP packets that
are to or from host B.
However, this will get both kinds of traffic (OOB and MPI). So if
you have a 2nd TCP network, use that for OOB and then the tcpdump
will capture only the MPI traffic. For example, you can force all
the OOB TCP traffic across the ib0 interface (vs. the default eth0
interface on my Linux machine) via:
a$ mpirun -np 2 --host a,b --mca btl tcp,self --mca oob_tcp_include
ib0 a.out
Make sense?
On May 18, 2007, at 1:36 PM, Code Master wrote:
> Suppose if I want to capture any packets for my openmpi program, if
> I can't filter packets by ports, then how can the sniffer tell
> which packets are from/to any processes of my penmpi program?
>
> On 5/19/07, Tim Prins <tprins_at_[hidden]> wrote: Open MPI uses
> TCP, and does not use any fixed ports. We use whatever ports the
> operating system gives us. At this time there is no way to specify
> what ports
> to use.
>
> Hope this helps,
>
> Tim
>
> On Friday 18 May 2007 05:19 am, Code Master wrote:
> > I run my openmpi-based application in a multi-node cluster.
> There is also
> > a sniffer computer (installed with wireshark) attached to a
> listener port
> > on the switch to sniff any packets.
> >
> > However I would like to know the protocol (UDP or TCP) as well as
> the ports
> > used by openmpi for interprocess communication so that wireshark
> can only
> > capture these packets.
> >
> > Thanks!
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
-- Jeff Squyres Cisco Systems