MPI Query: MPICH2

toltec · 2011-01-22 15:09:49

Hello All,

I've compiled CA with GNU & Intel v11.1 compilers and MPICH2. With mpi_nbcpu=1 the executable gives similar runtimes to a sequentially compiled one which is good news. However if I set mpi_nbcpu=2 runtime is MUCH longer (out of core, no swap used).

The .mess file shows:
-------------------------------------------------------
Version parallèle de Code_aster compilée avec MPI
Exécution sur le processeur de nom gary-desktop-LTS de rang 0
Nombre de processeurs utilisés 1

Version parallèle de Code_aster compilée avec MPI
Exécution sur le processeur de nom gary-desktop-LTS de rang 0
Nombre de processeurs utilisés 1
-------------------------------------------------------
Shouldn't the second rank show rang 1?

I have my suspicions about this because I have to use the openMPI option:
mpi_get_procid_cmd : echo $OMPI_MCA_orte_ess_vpid
to get the code to run as $PMI_RANK doesn't work. This seems odd!?

Any observations would be appreciated.

Gary

Last edited by toltec (2011-01-22 16:35:53)

mathieu.courtois · 2011-01-23 09:46:47

Hello,

mpi_get_procid_cmd is used by as_run to prepare the environment, start the execution...

This message is directly printed by the Code_Aster executable using the mpi library (see python.c) :

   MPI_Comm_size(MPI_COMM_WORLD, &nbproc);
   MPI_Comm_rank(MPI_COMM_WORLD, &me);
   MPI_Get_processor_name(processor_name, &namelen);
   printf("\n Version parallèle de Code_aster compilée avec MPI\n");
   printf(" Exécution sur le processeur de nom %s de rang %d\n",processor_name,me);
   printf(" Nombre de processeurs utilisés %d\n",nbproc);

Some ideas :
- check the mpirun command used by as_run (the mpi script should be dumped in the .e(rror) file of the job).
- try to reproduce the problem with a very simple executable...

MC

toltec · 2011-01-25 20:26:13

Hello MC,

Thanks for the reply. It seems that there is a problem in the way mpich2 is launching the processes. Setting mpi_nbcpu=4 (for eg) results in the following message repeated four times in the *o* temporay file:

"Version parallèle de Code_aster compilée avec MPI
Exécution sur le processeur de nom gary-desktop-LTS de rang 0
Nombre de processeurs utilisés 1"

So, four identical processes are launched it seems.

I have recompiled using the GNU compilers only (with MKL) and CA runs fine. The single message in the *o* file is:

"Version parallèle de Code_aster compilée avec MPI
Exécution sur le processeur de nom gary-desktop-LTS de rang 1
Nombre de processeurs utilisés 4 "

I suspect this has something to do with mpi_get_procid_cmd : echo $PMI_RANK not working,

I'll keep trying... I want the Intel MPI version to work and feel it must be close!

Gary

#1 2011-01-22 15:09:49

MPI Query: MPICH2

#2 2011-01-23 09:46:47

Re: MPI Query: MPICH2

#3 2011-01-25 20:26:13

Re: MPI Query: MPICH2

Board footer