From: Steven Truong (midair77_at_[hidden])
Date: 2007-05-09 14:19:18
Oh, no. I tried with ACML and had the same set of errors.
Steven.
On 5/9/07, Steven Truong <midair77_at_[hidden]> wrote:
> Hi, Kevin and all. I tried with the following:
>
> ./configure --prefix=/usr/local/openmpi-1.2.1 --disable-ipv6
> --with-tm=/usr/local/pbs --enable-mpirun-prefix-by-default
> --enable-mpi-f90 --with-threads=posix --enable-static
>
> and added the mpi.o in my VASP's makefile but i still got error.
>
> I forgot to mention that our environment has Intel MKL 9.0 or 8.1 and
> my machines are dual proc dual core Xeon 5130 .
>
> Well, I am going to try acml too.
>
> Attached is my makefile for VASP and I am not sure if I missed anything again.
>
> Thank you very much for all your helps.
>
> On 5/9/07, Steven Truong <midair77_at_[hidden]> wrote:
> > Thank Kevin and Brook for replying to my question. I am going to try
> > out what Kevin suggested.
> >
> > Steven.
> >
> > On 5/9/07, Kevin Radican <radicak_at_[hidden]> wrote:
> > > Hi,
> > >
> > > We use VASP 4.6 in parallel with opemmpi 1.1.2 without any problems on
> > > x86_64 with opensuse and compiled with gcc and Intel fortran and use
> > > torque PBS.
> > >
> > > I used standard configure to build openmpi something like
> > >
> > > ./configure --prefix=/usr/local --enable-static --with-threads
> > > --with-tm=/usr/local --with-libnuma
> > >
> > > I used the ACLM math lapack libs and built Blacs and Scalapack with them
> > > too.
> > >
> > > I attached my vasp makefile, I might of added
> > >
> > > mpi.o : mpi.F
> > > $(CPP)
> > > $(FC) -FR -lowercase -O0 -c $*$(SUFFIX)
> > >
> > > to the end of the make file, It doesn't look like it is in the example
> > > makefiles they give, but I compiled this a while ago.
> > >
> > > Hope this helps.
> > >
> > > Cheers,
> > > Kevin
> > >
> > >
> > >
> > >
> > >
> > > On Tue, 2007-05-08 at 19:18 -0700, Steven Truong wrote:
> > > > Hi, all. I am new to OpenMPI and after initial setup I tried to run
> > > > my app but got the followign errors:
> > > >
> > > > [node07.my.com:16673] *** An error occurred in MPI_Comm_rank
> > > > [node07.my.com:16673] *** on communicator MPI_COMM_WORLD
> > > > [node07.my.com:16673] *** MPI_ERR_COMM: invalid communicator
> > > > [node07.my.com:16673] *** MPI_ERRORS_ARE_FATAL (goodbye)
> > > > [node07.my.com:16674] *** An error occurred in MPI_Comm_rank
> > > > [node07.my.com:16674] *** on communicator MPI_COMM_WORLD
> > > > [node07.my.com:16674] *** MPI_ERR_COMM: invalid communicator
> > > > [node07.my.com:16674] *** MPI_ERRORS_ARE_FATAL (goodbye)
> > > > [node07.my.com:16675] *** An error occurred in MPI_Comm_rank
> > > > [node07.my.com:16675] *** on communicator MPI_COMM_WORLD
> > > > [node07.my.com:16675] *** MPI_ERR_COMM: invalid communicator
> > > > [node07.my.com:16675] *** MPI_ERRORS_ARE_FATAL (goodbye)
> > > > [node07.my.com:16676] *** An error occurred in MPI_Comm_rank
> > > > [node07.my.com:16676] *** on communicator MPI_COMM_WORLD
> > > > [node07.my.com:16676] *** MPI_ERR_COMM: invalid communicator
> > > > [node07.my.com:16676] *** MPI_ERRORS_ARE_FATAL (goodbye)
> > > > mpiexec noticed that job rank 2 with PID 16675 on node node07 exited
> > > > on signal 60 (Real-time signal 26).
> > > >
> > > > /usr/local/openmpi-1.2.1/bin/ompi_info
> > > > Open MPI: 1.2.1
> > > > Open MPI SVN revision: r14481
> > > > Open RTE: 1.2.1
> > > > Open RTE SVN revision: r14481
> > > > OPAL: 1.2.1
> > > > OPAL SVN revision: r14481
> > > > Prefix: /usr/local/openmpi-1.2.1
> > > > Configured architecture: x86_64-unknown-linux-gnu
> > > > Configured by: root
> > > > Configured on: Mon May 7 18:32:56 PDT 2007
> > > > Configure host: neptune.nanostellar.com
> > > > Built by: root
> > > > Built on: Mon May 7 18:40:28 PDT 2007
> > > > Built host: neptune.my.com
> > > > C bindings: yes
> > > > C++ bindings: yes
> > > > Fortran77 bindings: yes (all)
> > > > Fortran90 bindings: yes
> > > > Fortran90 bindings size: small
> > > > C compiler: gcc
> > > > C compiler absolute: /usr/bin/gcc
> > > > C++ compiler: g++
> > > > C++ compiler absolute: /usr/bin/g++
> > > > Fortran77 compiler: /opt/intel/fce/9.1.043/bin/ifort
> > > > Fortran77 compiler abs: /opt/intel/fce/9.1.043/bin/ifort
> > > > Fortran90 compiler: /opt/intel/fce/9.1.043/bin/ifort
> > > > Fortran90 compiler abs: /opt/intel/fce/9.1.043/bin/ifort
> > > > C profiling: yes
> > > > C++ profiling: yes
> > > > Fortran77 profiling: yes
> > > > Fortran90 profiling: yes
> > > > C++ exceptions: no
> > > > Thread support: posix (mpi: no, progress: no)
> > > > Internal debug support: no
> > > > MPI parameter check: runtime
> > > > Memory profiling support: no
> > > > Memory debugging support: no
> > > > libltdl support: yes
> > > > Heterogeneous support: yes
> > > > mpirun default --prefix: yes
> > > > MCA backtrace: execinfo (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA memory: ptmalloc2 (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA paffinity: linux (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA maffinity: first_use (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA maffinity: libnuma (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA timer: linux (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA installdirs: env (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA installdirs: config (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA allocator: basic (MCA v1.0, API v1.0, Component v1.0)
> > > > MCA allocator: bucket (MCA v1.0, API v1.0, Component v1.0)
> > > > MCA coll: basic (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA coll: self (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA coll: sm (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA coll: tuned (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA io: romio (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA mpool: rdma (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA mpool: sm (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA pml: cm (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA pml: ob1 (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA bml: r2 (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA rcache: vma (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA btl: self (MCA v1.0, API v1.0.1, Component v1.2.1)
> > > > MCA btl: sm (MCA v1.0, API v1.0.1, Component v1.2.1)
> > > > MCA btl: tcp (MCA v1.0, API v1.0.1, Component v1.0)
> > > > MCA topo: unity (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA osc: pt2pt (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA errmgr: hnp (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA errmgr: orted (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA errmgr: proxy (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA gpr: null (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA gpr: proxy (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA gpr: replica (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA iof: proxy (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA iof: svc (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA ns: proxy (MCA v1.0, API v2.0, Component v1.2.1)
> > > > MCA ns: replica (MCA v1.0, API v2.0, Component v1.2.1)
> > > > MCA oob: tcp (MCA v1.0, API v1.0, Component v1.0)
> > > > MCA ras: dash_host (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA ras: gridengine (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA ras: localhost (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA ras: slurm (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA ras: tm (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA rds: hostfile (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA rds: proxy (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA rds: resfile (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA rmaps: round_robin (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA rmgr: proxy (MCA v1.0, API v2.0, Component v1.2.1)
> > > > MCA rmgr: urm (MCA v1.0, API v2.0, Component v1.2.1)
> > > > MCA rml: oob (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA pls: gridengine (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA pls: proxy (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA pls: rsh (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA pls: slurm (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA pls: tm (MCA v1.0, API v1.3, Component v1.2.1)
> > > > MCA sds: env (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA sds: pipe (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA sds: seed (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA sds: singleton (MCA v1.0, API v1.0, Component v1.2.1)
> > > > MCA sds: slurm (MCA v1.0, API v1.0, Component v1.2.1)
> > > >
> > > > As you can see, I used Gnu gcc and g++ with Intel Fortran Compiler to
> > > > compile Open MPI and I am not sure if there are any special flags that
> > > > I need to have.
> > > > ./configure --prefix=/usr/local/openmpi-1.2.1 --disable-ipv6
> > > > --with-tm=/usr/local/pbs --enable-mpirun-prefix-by-default
> > > > --enable-mpi-f90
> > > >
> > > > After getting mpif90, I compiled my application (VASP) with this new
> > > > parellel compiler but then I could not run it through PBS.
> > > >
> > > > #PBS -N Pt.CO.bridge.25ML
> > > > ### Set the number of nodes that will be used. Ensure
> > > > ### that the number "nodes" matches with the need of your job
> > > > ### DO NOT MODIFY THE FOLLOWING LINE FOR SINGLE-PROCESSOR JOBS!
> > > > #PBS -l nodes=node07:ppn=4
> > > > #PBS -l walltime=96:00:00
> > > > ##PBS -M asit_at_[hidden]
> > > > #PBS -m abe
> > > > export NPROCS=`wc -l $PBS_NODEFILE |gawk '//{print $1}'`
> > > > echo $NPROCS
> > > > echo The master node of this job is `hostname`
> > > > echo The working directory is `echo $PBS_O_WORKDIR`
> > > > echo The node file is $PBS_NODEFILE
> > > > echo This job runs on the following $NPROCS nodes:
> > > > echo `cat $PBS_NODEFILE`
> > > > echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-"
> > > > echo
> > > > echo command to EXE:
> > > > echo
> > > > echo
> > > > cd $PBS_O_WORKDIR
> > > >
> > > > echo "cachesize=4000 mpiblock=500 npar=4 procgroup=4 mkl ompi"
> > > >
> > > > date
> > > > /usr/local/openmpi-1.2.1/bin/mpiexec -mca mpi_paffinity_alone 1 -np
> > > > $NPROCS /hom e/struong/bin/vaspmpi_mkl_ompi >"$PBS_JOBID".out
> > > > date
> > > > ------------
> > > >
> > > > My environment is CentOS 4.4 x86_64, Intel Xeon, Torque, Maui.
> > > >
> > > > Could somebody here tell me what I missed or did incorrectly?
> > > >
> > > > Thank you very much.
> > > > _______________________________________________
> > > > users mailing list
> > > > users_at_[hidden]
> > > > http://www.open-mpi.org/mailman/listinfo.cgi/users
> > > >
> > >
> > > _______________________________________________
> > > users mailing list
> > > users_at_[hidden]
> > > http://www.open-mpi.org/mailman/listinfo.cgi/users
> > >
> > >
> >
>
>