[Users] Unable to compile ET with OpenMPI 2.1.0 in cluster

Roland Haas rhaas at illinois.edu
Thu Dec 16 09:50:40 CST 2021

Hello Shamim Haque,

> I have done it successfully once with OpenMPI-x86_64 in our cluster, but it
> turns out this MPI is only available in the login node and not available on
> computing nodes, hence, I am not able to use it once a job is submitted in
> the queue.
This is something you should bring up with the cluster admins. Having
MPI only on the login nodes but not on the compute nodes makes no sense
since MPI is an inter-node communication library.

> I could not figure out the issue here. It seems
> */usr/include/openmpi-x86_64/openmpi/ompi/mpi
> *is being taken by ET from somewhere and it is not taking OpenMPI 2.1.0.
My guess based on the error message being about MPI C++ bindings is
that the issue is similar to this recent ET ticket:


namely you have to manually set MPI_LIBS in you option list to include
mpi_cxx (or the equivalent name for your cluster).

You can find out the other libraries to include by quetying mpic++:

mpic++ -showme:link

which will output the 


setting (the directory after -L) and the libraries afterwards (you need
to remove the -l to get the actual name) and use

mpic++ -showme:compile

to similarly get the directory to put into the 


variable. You should then set MPI_DIR to the directory containing

Ie if "which mpic++" returns /usr/local/openmpi/bin/mpic++ you would
use /usr/local/openmpi

Note that OpenMPI 2.1 is somewhat old (from 2019) and no longer among
the supported releases by the OpenMPI team, so you may also ask the
admins if there is any newer version available.


My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20211216/171f2f09/attachment.bin 

More information about the Users mailing list