[Users] Unable to compile ET with OpenMPI 2.1.0 in cluster

[Shamim Haque]

Hi Roland,

Thanks for the help. Turns out libmpi_cxx.so was not present in this
particular openmpi library. So I compiled with another version of openmpi
which has this file, and now ET seems to be working fine after a
successful compilation. Meanwhile, I am working out this issue with cluster
admins to resolve the issue issue with openmpi-x86_64.

Regards
Shamim Haque
Senior Research Fellow (SRF)
Department of Physics
IISER Bhopal

ᐧ

On Thu, Dec 16, 2021 at 9:20 PM Roland Haas <rhaas at illinois.edu> wrote:

> Hello Shamim Haque,
>
> > I have done it successfully once with OpenMPI-x86_64 in our cluster, but
> it
> > turns out this MPI is only available in the login node and not available
> on
> > computing nodes, hence, I am not able to use it once a job is submitted
> in
> > the queue.
> This is something you should bring up with the cluster admins. Having
> MPI only on the login nodes but not on the compute nodes makes no sense
> since MPI is an inter-node communication library.
>
> > I could not figure out the issue here. It seems
> > */usr/include/openmpi-x86_64/openmpi/ompi/mpi
> > *is being taken by ET from somewhere and it is not taking OpenMPI 2.1.0.
> My guess based on the error message being about MPI C++ bindings is
> that the issue is similar to this recent ET ticket:
>
>
> https://bitbucket.org/einsteintoolkit/tickets/issues/2578/kranc-cactus-and-mpi-linking-issues
>
> namely you have to manually set MPI_LIBS in you option list to include
> mpi_cxx (or the equivalent name for your cluster).
>
> You can find out the other libraries to include by quetying mpic++:
>
> mpic++ -showme:link
>
> which will output the
>
> MPI_LIB_DIRS
>
> setting (the directory after -L) and the libraries afterwards (you need
> to remove the -l to get the actual name) and use
>
> mpic++ -showme:compile
>
> to similarly get the directory to put into the
>
> MPI_INC_DIRS
>
> variable. You should then set MPI_DIR to the directory containing
> bin/mpic++
>
> Ie if "which mpic++" returns /usr/local/openmpi/bin/mpic++ you would
> use /usr/local/openmpi
>
> Note that OpenMPI 2.1 is somewhat old (from 2019) and no longer among
> the supported releases by the OpenMPI team, so you may also ask the
> admins if there is any newer version available.
>
> Yours,
> Roland
>
> --
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://pgp.mit.edu .
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20220112/6108ee00/attachment.html