[Users] Unable to compile ET with OpenMPI 2.1.0 in cluster

[Shamim Haque]

Hello all,

I am trying to install ET with OpenMPI 2.1.0 but it seems to be causing
issues.

I have done it successfully once with OpenMPI-x86_64 in our cluster, but it
turns out this MPI is only available in the login node and not available on
computing nodes, hence, I am not able to use it once a job is submitted in
the queue.

I could find OpenMPI 2.1.0 being available on computing nodes, so I was
trying to compile with this version. I created a module file to load this
module before installation. I could see the following in the terminal,
which I guess, ET could recognise OpenMPI 2.1.0:

Running configuration script for thorn MPI:
MPI selected, but MPI_DIR is not set. Computing settings...
Found MPI compiler wrapper at /opt/mpi/openmpi/2.1.0/bin/mpic++!
Successfully configured MPI.
Finished running configuration script for thorn MPI.

Later, the progress stops with the following error:

/home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):
In function `MPI::Op::Init(void (*)(void const*, void*, int, MPI::Datatype
const&), bool)':
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/op_inln.h:121: undefined
reference to `ompi_mpi_cxx_op_intercept'
/home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):
In function `Intracomm':
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm_inln.h:23:
undefined reference to `MPI::Comm::Comm()'
/home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):
In function `Intracomm':
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined
reference to `MPI::Comm::Comm()'
/home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):
In function `Cartcomm':
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined
reference to `MPI::Comm::Comm()'
/home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):
In function `Graphcomm':
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined
reference to `MPI::Comm::Comm()'
/home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):
In function `Intracomm':
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined
reference to `MPI::Comm::Comm()'
/home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25:
more undefined references to `MPI::Comm::Comm()' follow
/home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):(.rodata._ZTVN3MPI8DatatypeE[_ZTVN3MPI8DatatypeE]+0x78):
undefined reference to `MPI::Datatype::Free()'
/home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):(.rodata._ZTVN3MPI3WinE[_ZTVN3MPI3WinE]+0x48):
undefined reference to `MPI::Win::Free()'
/home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CarpetLib.a(backtrace.cc.o):
In function `MPI::Intracomm::Intracomm()':
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined
reference to `MPI::Comm::Comm()'
/home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CarpetLib.a(backtrace.cc.o):
In function `MPI::Intracomm::Intracomm(ompi_communicator_t*)':
/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm_inln.h:23:
undefined reference to `MPI::Comm::Comm()'
collect2: error: ld returned 1 exit status
make[1]: *** [/home2/mallick/ET3/Cactus/exe/cactus_sim] Error 1
make: *** [sim] Error 2

I could not figure out the issue here. It seems
*/usr/include/openmpi-x86_64/openmpi/ompi/mpi
*is being taken by ET from somewhere and it is not taking OpenMPI 2.1.0.

Some suggestions would be very helpful. Thank you.

Regards
Shamim Haque
Senior Research Fellow (SRF)
Department of Physics
IISER Bhopal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20211211/dc980f69/attachment.html