From: Adrian Knoth (adi_at_[hidden])
Date: 2007-05-18 17:25:13
On Sat, May 19, 2007 at 08:36:50AM +1200, Code Master wrote:
> Suppose if I want to capture any packets for my openmpi program, if I
> can't filter packets by ports, then how can the sniffer tell which packets
> are from/to any processes of my penmpi program?
You first have to distinguish between ORTE and OMPI communication.
ORTE communication (wire up) is a little easier to capture, because you
can extract the target port from the command line arguments.
Take a typical process launch via ssh:
0.0.0;tcp://141.35.14.189:57985;tcp://192.168.1.1:57985
57985 is the listen port of the headnode process.
This information should also be available via ps or any other tool to
query the process list (i.e. top)
netstat can also be used:
adi_at_ipc654:~$ orterun -host amun3 netstat -tpn
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 192.168.1.132:48580 192.168.1.1:41753 ESTABLISHED25377/orted
(output reduced to the relevant lines).
The actual MPI communication is harder to capture. Every host has its
own listen port. Given that, you'll have to use the BTL/TCP debug output
and grep for the relevant strings.
To enable this output, change WANT_PEER_DUMP from 0 to 1 in
btl_tcp_endpoint.c and make sure you compile with --enable-debug.
If you don't want to parse dynamic ports or you don't want to lower your
MPI performance due to --enable-debug, you can easily change the code to
use a static port:
For the BTL, see btl_tcp_component.c:
Line 522, in function mca_btl_tcp_component_create_listen:
if ((error = getaddrinfo(NULL, "0", &hints, &res))) {
change the "0" to your desired port, or if you compile without IPv6,
change line 546:
inaddr.sin_port = 0;
For the OOB (wire-up connection), change oob_tcp.c:445 and/or
oob_tcp.c:469, the code is absolutely the same.
You can optionally create two parameters (one for oob_tcp and one for btl_tcp)
to set the ports from your mca-params.conf. Creating parameters isn't
that hard, you'll find examples inside the code (i.e.
btl_tcp_disable_family).
HTH
-- Cluster and Metacomputing Working Group Friedrich-Schiller-Universität Jena, Germany private: http://adi.thur.de