From: Brock Palen (brockp_at_[hidden])
Date: 2007-05-08 22:26:06
Steven,
We run vasp on both Linux (PGI compilers) and Max OSX (xlf) I am sad
to announce that VASP does not work with openMPI last I tried
(1.1.1) With the errors you reported are the same I saw. VASP for
the time (version 4) Works only with Lam and MPICH-1.
If you have insight into the vasp devs that both those projects are
un-maintined it would be a huge help for us all!
Again though if any Fortran guru has had time to find out how to make
VASP work with OMPI Please contact us all right away!
Brock Palen
Center for Advanced Computing
brockp_at_[hidden]
(734)936-1985
On May 8, 2007, at 10:18 PM, Steven Truong wrote:
> Hi, all. I am new to OpenMPI and after initial setup I tried to run
> my app but got the followign errors:
>
> [node07.my.com:16673] *** An error occurred in MPI_Comm_rank
> [node07.my.com:16673] *** on communicator MPI_COMM_WORLD
> [node07.my.com:16673] *** MPI_ERR_COMM: invalid communicator
> [node07.my.com:16673] *** MPI_ERRORS_ARE_FATAL (goodbye)
> [node07.my.com:16674] *** An error occurred in MPI_Comm_rank
> [node07.my.com:16674] *** on communicator MPI_COMM_WORLD
> [node07.my.com:16674] *** MPI_ERR_COMM: invalid communicator
> [node07.my.com:16674] *** MPI_ERRORS_ARE_FATAL (goodbye)
> [node07.my.com:16675] *** An error occurred in MPI_Comm_rank
> [node07.my.com:16675] *** on communicator MPI_COMM_WORLD
> [node07.my.com:16675] *** MPI_ERR_COMM: invalid communicator
> [node07.my.com:16675] *** MPI_ERRORS_ARE_FATAL (goodbye)
> [node07.my.com:16676] *** An error occurred in MPI_Comm_rank
> [node07.my.com:16676] *** on communicator MPI_COMM_WORLD
> [node07.my.com:16676] *** MPI_ERR_COMM: invalid communicator
> [node07.my.com:16676] *** MPI_ERRORS_ARE_FATAL (goodbye)
> mpiexec noticed that job rank 2 with PID 16675 on node node07 exited
> on signal 60 (Real-time signal 26).
>
> /usr/local/openmpi-1.2.1/bin/ompi_info
> Open MPI: 1.2.1
> Open MPI SVN revision: r14481
> Open RTE: 1.2.1
> Open RTE SVN revision: r14481
> OPAL: 1.2.1
> OPAL SVN revision: r14481
> Prefix: /usr/local/openmpi-1.2.1
> Configured architecture: x86_64-unknown-linux-gnu
> Configured by: root
> Configured on: Mon May 7 18:32:56 PDT 2007
> Configure host: neptune.nanostellar.com
> Built by: root
> Built on: Mon May 7 18:40:28 PDT 2007
> Built host: neptune.my.com
> C bindings: yes
> C++ bindings: yes
> Fortran77 bindings: yes (all)
> Fortran90 bindings: yes
> Fortran90 bindings size: small
> C compiler: gcc
> C compiler absolute: /usr/bin/gcc
> C++ compiler: g++
> C++ compiler absolute: /usr/bin/g++
> Fortran77 compiler: /opt/intel/fce/9.1.043/bin/ifort
> Fortran77 compiler abs: /opt/intel/fce/9.1.043/bin/ifort
> Fortran90 compiler: /opt/intel/fce/9.1.043/bin/ifort
> Fortran90 compiler abs: /opt/intel/fce/9.1.043/bin/ifort
> C profiling: yes
> C++ profiling: yes
> Fortran77 profiling: yes
> Fortran90 profiling: yes
> C++ exceptions: no
> Thread support: posix (mpi: no, progress: no)
> Internal debug support: no
> MPI parameter check: runtime
> Memory profiling support: no
> Memory debugging support: no
> libltdl support: yes
> Heterogeneous support: yes
> mpirun default --prefix: yes
> MCA backtrace: execinfo (MCA v1.0, API v1.0, Component
> v1.2.1)
> MCA memory: ptmalloc2 (MCA v1.0, API v1.0, Component
> v1.2.1)
> MCA paffinity: linux (MCA v1.0, API v1.0, Component v1.2.1)
> MCA maffinity: first_use (MCA v1.0, API v1.0, Component
> v1.2.1)
> MCA maffinity: libnuma (MCA v1.0, API v1.0, Component
> v1.2.1)
> MCA timer: linux (MCA v1.0, API v1.0, Component v1.2.1)
> MCA installdirs: env (MCA v1.0, API v1.0, Component v1.2.1)
> MCA installdirs: config (MCA v1.0, API v1.0, Component
> v1.2.1)
> MCA allocator: basic (MCA v1.0, API v1.0, Component v1.0)
> MCA allocator: bucket (MCA v1.0, API v1.0, Component v1.0)
> MCA coll: basic (MCA v1.0, API v1.0, Component v1.2.1)
> MCA coll: self (MCA v1.0, API v1.0, Component v1.2.1)
> MCA coll: sm (MCA v1.0, API v1.0, Component v1.2.1)
> MCA coll: tuned (MCA v1.0, API v1.0, Component v1.2.1)
> MCA io: romio (MCA v1.0, API v1.0, Component v1.2.1)
> MCA mpool: rdma (MCA v1.0, API v1.0, Component v1.2.1)
> MCA mpool: sm (MCA v1.0, API v1.0, Component v1.2.1)
> MCA pml: cm (MCA v1.0, API v1.0, Component v1.2.1)
> MCA pml: ob1 (MCA v1.0, API v1.0, Component v1.2.1)
> MCA bml: r2 (MCA v1.0, API v1.0, Component v1.2.1)
> MCA rcache: vma (MCA v1.0, API v1.0, Component v1.2.1)
> MCA btl: self (MCA v1.0, API v1.0.1, Component
> v1.2.1)
> MCA btl: sm (MCA v1.0, API v1.0.1, Component v1.2.1)
> MCA btl: tcp (MCA v1.0, API v1.0.1, Component v1.0)
> MCA topo: unity (MCA v1.0, API v1.0, Component v1.2.1)
> MCA osc: pt2pt (MCA v1.0, API v1.0, Component v1.2.1)
> MCA errmgr: hnp (MCA v1.0, API v1.3, Component v1.2.1)
> MCA errmgr: orted (MCA v1.0, API v1.3, Component v1.2.1)
> MCA errmgr: proxy (MCA v1.0, API v1.3, Component v1.2.1)
> MCA gpr: null (MCA v1.0, API v1.0, Component v1.2.1)
> MCA gpr: proxy (MCA v1.0, API v1.0, Component v1.2.1)
> MCA gpr: replica (MCA v1.0, API v1.0, Component
> v1.2.1)
> MCA iof: proxy (MCA v1.0, API v1.0, Component v1.2.1)
> MCA iof: svc (MCA v1.0, API v1.0, Component v1.2.1)
> MCA ns: proxy (MCA v1.0, API v2.0, Component v1.2.1)
> MCA ns: replica (MCA v1.0, API v2.0, Component
> v1.2.1)
> MCA oob: tcp (MCA v1.0, API v1.0, Component v1.0)
> MCA ras: dash_host (MCA v1.0, API v1.3, Component
> v1.2.1)
> MCA ras: gridengine (MCA v1.0, API v1.3, Component
> v1.2.1)
> MCA ras: localhost (MCA v1.0, API v1.3, Component
> v1.2.1)
> MCA ras: slurm (MCA v1.0, API v1.3, Component v1.2.1)
> MCA ras: tm (MCA v1.0, API v1.3, Component v1.2.1)
> MCA rds: hostfile (MCA v1.0, API v1.3, Component
> v1.2.1)
> MCA rds: proxy (MCA v1.0, API v1.3, Component v1.2.1)
> MCA rds: resfile (MCA v1.0, API v1.3, Component
> v1.2.1)
> MCA rmaps: round_robin (MCA v1.0, API v1.3,
> Component v1.2.1)
> MCA rmgr: proxy (MCA v1.0, API v2.0, Component v1.2.1)
> MCA rmgr: urm (MCA v1.0, API v2.0, Component v1.2.1)
> MCA rml: oob (MCA v1.0, API v1.0, Component v1.2.1)
> MCA pls: gridengine (MCA v1.0, API v1.3, Component
> v1.2.1)
> MCA pls: proxy (MCA v1.0, API v1.3, Component v1.2.1)
> MCA pls: rsh (MCA v1.0, API v1.3, Component v1.2.1)
> MCA pls: slurm (MCA v1.0, API v1.3, Component v1.2.1)
> MCA pls: tm (MCA v1.0, API v1.3, Component v1.2.1)
> MCA sds: env (MCA v1.0, API v1.0, Component v1.2.1)
> MCA sds: pipe (MCA v1.0, API v1.0, Component v1.2.1)
> MCA sds: seed (MCA v1.0, API v1.0, Component v1.2.1)
> MCA sds: singleton (MCA v1.0, API v1.0, Component
> v1.2.1)
> MCA sds: slurm (MCA v1.0, API v1.0, Component v1.2.1)
>
> As you can see, I used Gnu gcc and g++ with Intel Fortran Compiler to
> compile Open MPI and I am not sure if there are any special flags that
> I need to have.
> ./configure --prefix=/usr/local/openmpi-1.2.1 --disable-ipv6
> --with-tm=/usr/local/pbs --enable-mpirun-prefix-by-default
> --enable-mpi-f90
>
> After getting mpif90, I compiled my application (VASP) with this new
> parellel compiler but then I could not run it through PBS.
>
> #PBS -N Pt.CO.bridge.25ML
> ### Set the number of nodes that will be used. Ensure
> ### that the number "nodes" matches with the need of your job
> ### DO NOT MODIFY THE FOLLOWING LINE FOR SINGLE-PROCESSOR JOBS!
> #PBS -l nodes=node07:ppn=4
> #PBS -l walltime=96:00:00
> ##PBS -M asit_at_[hidden]
> #PBS -m abe
> export NPROCS=`wc -l $PBS_NODEFILE |gawk '//{print $1}'`
> echo $NPROCS
> echo The master node of this job is `hostname`
> echo The working directory is `echo $PBS_O_WORKDIR`
> echo The node file is $PBS_NODEFILE
> echo This job runs on the following $NPROCS nodes:
> echo `cat $PBS_NODEFILE`
> echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-"
> echo
> echo command to EXE:
> echo
> echo
> cd $PBS_O_WORKDIR
>
> echo "cachesize=4000 mpiblock=500 npar=4 procgroup=4 mkl ompi"
>
> date
> /usr/local/openmpi-1.2.1/bin/mpiexec -mca mpi_paffinity_alone 1 -np
> $NPROCS /hom e/struong/bin/vaspmpi_mkl_ompi >"$PBS_JOBID".out
> date
> ------------
>
> My environment is CentOS 4.4 x86_64, Intel Xeon, Torque, Maui.
>
> Could somebody here tell me what I missed or did incorrectly?
>
> Thank you very much.
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>