Re: [Users] [External - Warning: Suspect Attachment] null Re: mpi.h 9% - Users

25 Mar 2018


      Hello DG,
it looks like you are trying to use the system mpicc to compile OpenMPI
in Cactus' ExternalLibraries.
This is a very odd thing to do and I am not surprised that you would
fail attempting this.
Instead why not set MPI_DIR = <PATH-TO-DIRECTORY-WHERE-MPICC-IC>/.. ie
the parent directory of the system MPI's "bin" directory? Leave the
other MPI settings blank and Cactus should obtain the required settings
from the system MPI that it will find in MPI_DIR.
Yours,
Roland
...
Hello,
Using mpicc, etc… seems to work a lot better but I’m still having a couple issues.  When I use the cluster’s mpi, I get the error
ld: cannot find -lmpi_usempif08
make[2]: *** [/home/dgarriso/Cactus/exe/cactus_fixedmhd] Error 1
make[1]: *** [fixedmhd] Error 2
make: *** [fixedmhd-rebuild] Error 2
When I try to build the Cactus mpi, it fails
Making all in libevent
Making all in .
libtool: compile:  mpiicc -DHAVE_CONFIG_H -I. -I./compat -I./include -I./include -DCCTK_DISABLE_OMP_COLLAPSE -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED -std=c99 -Os -ip -pthread -no-prec-div -Wall -fno-strict-aliasing -MT event.lo -MD -MP -MF .deps/event.Tpo -c event.c -o event.o
libtool: compile:  mpiicc -DHAVE_CONFIG_H -I. -I./compat -I./include -I./include -DCCTK_DISABLE_OMP_COLLAPSE -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED -std=c99 -Os -ip -pthread -no-prec-div -Wall -fno-strict-aliasing -MT evthread.lo -MD -MP -MF .deps/evthread.Tpo -c evthread.c -o evthread.o
libtool: compile:  mpiicc -DHAVE_CONFIG_H -I. -I./compat -I./include -I./include -DCCTK_DISABLE_OMP_COLLAPSE -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED -std=c99 -Os -ip -pthread -no-prec-div -Wall -fno-strict-aliasing -MT evmap.lo -MD -MP -MF .deps/evmap.Tpo -c evmap.c -o evmap.o
libtool: compile:  mpiicc -DHAVE_CONFIG_H -I. -I./compat -I./include -I./include -DCCTK_DISABLE_OMP_COLLAPSE -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED -std=c99 -Os -ip -pthread -no-prec-div -Wall -fno-strict-aliasing -MT log.lo -MD -MP -MF .deps/log.Tpo -c log.c -o log.o
libtool: compile:  mpiicc -DHAVE_CONFIG_H -I. -I./compat -I./include -I./include -DCCTK_DISABLE_OMP_COLLAPSE -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED -std=c99 -Os -ip -pthread -no-prec-div -Wall -fno-strict-aliasing -MT evutil.lo -MD -MP -MF .deps/evutil.Tpo -c evutil.c -o evutil.o
libtool: compile:  mpiicc -DHAVE_CONFIG_H -I. -I./compat -I./include -I./include -DCCTK_DISABLE_OMP_COLLAPSE -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED -std=c99 -Os -ip -pthread -no-prec-div -Wall -fno-strict-aliasing -MT evutil_rand.lo -MD -MP -MF .deps/evutil_rand.Tpo -c evutil_rand.c -o evutil_rand.o
libtool: compile:  mpiicc -DHAVE_CONFIG_H -I. -I./compat -I./include -I./include -DCCTK_DISABLE_OMP_COLLAPSE -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED -std=c99 -Os -ip -pthread -no-prec-div -Wall -fno-strict-aliasing -MT strlcpy.lo -MD -MP -MF .deps/strlcpy.Tpo -c strlcpy.c -o strlcpy.o
libtool: compile:  mpiicc -DHAVE_CONFIG_H -I. -I./compat -I./include -I./include -DCCTK_DISABLE_OMP_COLLAPSE -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED -std=c99 -Os -ip -pthread -no-prec-div -Wall -fno-strict-aliasing -MT select.lo -MD -MP -MF .deps/select.Tpo -c select.c -o select.o
libtool: compile:  mpiicc -DHAVE_CONFIG_H -I. -I./compat -I./include -I./include -DCCTK_DISABLE_OMP_COLLAPSE -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED -std=c99 -Os -ip -pthread -no-prec-div -Wall -fno-strict-aliasing -MT poll.lo -MD -MP -MF .deps/poll.Tpo -c poll.c -o poll.o
libtool: compile:  mpiicc -DHAVE_CONFIG_H -I. -I./compat -I./include -I./include -DCCTK_DISABLE_OMP_COLLAPSE -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED -std=c99 -Os -ip -pthread -no-prec-div -Wall -fno-strict-aliasing -MT epoll.lo -MD -MP -MF .deps/epoll.Tpo -c epoll.c -o epoll.o
libtool: compile:  mpiicc -DHAVE_CONFIG_H -I. -I./compat -I./include -I./include -DCCTK_DISABLE_OMP_COLLAPSE -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED -std=c99 -Os -ip -pthread -no-prec-div -Wall -fno-strict-aliasing -MT signal.lo -MD -MP -MF .deps/signal.Tpo -c signal.c -o signal.o
signal.c(135): error: identifier "NSIG" is undefined
   int ncaught[NSIG];
               ^
signal.c(253): error: identifier "SA_RESTART" is undefined
   sa.sa_flags |= SA_RESTART;
                  ^
signal.c(281): error: identifier "NSIG" is undefined
   EVUTIL_ASSERT(evsignal >= 0 && evsignal < NSIG);
   ^
signal.c(355): error: identifier "NSIG" is undefined
   EVUTIL_ASSERT(evsignal >= 0 && evsignal < NSIG);
   ^
signal.c(409): error: identifier "NSIG" is undefined
   for (i = 0; i < NSIG; ++i) {
                   ^
compilation aborted for signal.c (code 2)
make[10]: *** [signal.lo] Error 1
make[9]: *** [all-recursive] Error 1
make[8]: *** [all] Error 2
make[7]: *** [all-recursive] Error 1
make[6]: *** [all-recursive] Error 1
make[5]: *** [all-recursive] Error 1
Died at /home/dgarriso/Cactus/arrangements/ExternalLibraries/MPI/src/build.pl line 74.
make[4]: *** [/home/dgarriso/Cactus/configs/fixedmhd/scratch/done/MPI] Error 17
make[3]: *** [make.checked] Error 2
make[2]: *** [/home/dgarriso/Cactus/configs/fixedmhd/lib/libthorn_MPI.a] Error 2
make[1]: *** [fixedmhd] Error 2
make: *** [fixedmhd-rebuild] Error 2
Is this an issue with the cluster mpi installation or with how cactus works with the compilers?  I am using Intel compilers from 2015 (intel/ics-2015-64-bit), and openmpi version 1.6.5 on the cluster compiled with intel compilers.
-DG
...
On Mar 21, 2018, at 9:52 AM, Roland Haas rhaas@illinois.edu wrote:
Hello DG,
hmm that all look fine to me. From the content of
configs/sim/bindings/Configuration/Capabilities/make.MPI.defn  it is using a system installed MPI (openmpi-1.6.5 in /share/apps) which makes sense given that you pass MPI_DIR=/share/apps/openmpi-1.6.5 in your config-info (which is basically a copy of your option list). So MPI_DIR=BUILD was not used in the end (through an oversight or something else I guess).
Since you are passing an explicit
MPI_INC_DIRS=/share/apps/openmpi-1.6.5/include
I would definitely myself try and see if indeed there is a file
/share/apps/openmpi-1.6.5/include/mpi.h
and that this file is readable to your user account (eg does
less /share/apps/openmpi-1.6.5/include/mpi.h work?).
This is just in case /share/apps/openmpi-1.6.5 was eg retired by the
sysadmins.
Yours,
Roland
...
Hello,
Please see below.  I’m using UH’s Maxwell cluster and don’t use simfactory.
Thanks,
-DG
...
On Mar 20, 2018, at 11:47 AM, Roland Haas roland.haas@physics.gatech.edu wrote:
Hello DG,
it could be that something failed while MPI was compiled. Would you mind attaching the following files from inside of your Cactus tree, please?
configs/sim/bindings/Configuration/Capabilities/make.MPI.defn
INC_DIRS +=  $(MPI_INC_DIRS)
INC_DIRS_F +=  $(MPI_LIB_DIRS)
MPI_BUILD       = 
MPI_INSTALL_DIR = 
HWLOC_DIR       = 
CCTK_MPI     = 1
MPI_DIR      = /share/apps/openmpi-1.6.5
MPI_INC_DIRS = /share/apps/openmpi-1.6.5/include
MPI_LIB_DIRS = /share/apps/openmpi-1.6.5/lib
MPI_LIBS     = mpi mpi_cxx  open-rte open-pal dl nsl util
HAVE_CAPABILITY_MPI = 1
...
configs/sim/bindings/Configuration/Capabilities/make.HDF5.defn
include /home/dgarriso/Cactus/configs/fixedmhd/bindings/Configuration/Capabilities/make.MPI.defn
include /home/dgarriso/Cactus/configs/fixedmhd/bindings/Configuration/Capabilities/make.ZLIB.defn
INC_DIRS +=  $(HDF5_INC_DIRS)
INC_DIRS_F +=  $(HDF5_INC_DIRS)
HDF5_BUILD          = 
HDF5_ENABLE_CXX     = no
HDF5_ENABLE_FORTRAN = yes
LIBSZ_DIR           = 
LIBZ_DIR            = 
HDF5_INSTALL_DIR    = 
HDF5_DIR            = /share/apps/hdf5-1.8.12
HDF5_ENABLE_CXX     = no
HDF5_ENABLE_FORTRAN = yes
HDF5_INC_DIRS       = /share/apps/hdf5-1.8.12/lib /share/apps/hdf5-1.8.12/include  /share/apps/openmpi-1.6.5/include 
HDF5_LIB_DIRS       = /share/apps/hdf5-1.8.12/lib   /share/apps/openmpi-1.6.5/lib 
HDF5_LIBS           =  hdf5hl_fortran hdf5_fortran hdf5_hl hdf5 z mpi mpi_cxx  open-rte open-pal dl nsl util m z
HAVE_CAPABILITY_HDF5 = 1
...
configs/sim/bindings/Configuration/Capabilities/make.HDF5.deps
include /home/dgarriso/Cactus/configs/fixedmhd/bindings/Configuration/Capabilities/make.MPI.deps
include /home/dgarriso/Cactus/configs/fixedmhd/bindings/Configuration/Capabilities/make.ZLIB.deps
...
as well as
confis/sim/config-info (as well as the option list that you used)?
# CONFIGURATION  : fixedmhd
# CONFIG-DATE    : Mon Mar 19 01:07:33 2018 (GMT)
# CONFIG-HOST    : cusco.hpcc.uh.edu
# CONFIG-STATUS  : 0
# CONFIG-OPTIONS :
CC=icc
CPPFLAGS=-DCCTK_DISABLE_OMP_COLLAPSE
CXX=icpc
CXX_OPTIMISE_FLAGS=-Os -ip -pthread -no-prec-div
C_OPTIMISE_FLAGS=-Os -ip -pthread -no-prec-div
F77=ifort
F77_OPTIMISE_FLAGS=-Os -ip -no-prec-div
F90=ifort
F90_OPTIMISE_FLAGS=-Os -ip -no-prec-div
FFLAGS=-O3 -fast -Mipa=fast,inline -tp=x64
FFTW=yes
FFTW_DIR=/home/dgarriso/fftw3
FFTW_LIBS=fftw3_threads fftw3_mpi $(MPI_LIBS) fftw3 m
FPPFLAGS=-DCCTK_DISABLE_OMP_COLLAPSE
HDF5_DIR=/share/apps/hdf5-1.8.12
LAPACK=yes
LAPACK_DIR=/home/dgarriso/ifort64/lib
LAPACK_LIBS=acml ifcoremt_pic imf irc svml
MPI_DIR=/share/apps/openmpi-1.6.5
MPI_INC_DIRS=/share/apps/openmpi-1.6.5/include
MPI_LIBS=mpi mpi_cxx  open-rte open-pal dl nsl util
MPI_LIB_DIRS=/share/apps/openmpi-1.6.5/lib
OPTIMISE=yes
PTHREADS=no
VERBOSE=yes
...
If possible also the full output of:
export VERBOSE=yes
simfactory/bin/sim build 2>&1 | tee make.log
or
export VERBOSE=yes
make sim 2>&1 | tee make.log
(depending on whether you used simfactory or not), please?
Yours,
Roland
...
Hello,
Every time I try to compile my code I get the following error.
COMPILING arrangements/CactusPUGHIO/IOHDF5Util/src/Startup.c
/share/apps/hdf5-1.8.12/include/H5public.h(61): catastrophic error: cannot open source file "mpi.h"
#   include <mpi.h>
This is new as of the most recent versions of Cactus.  Previous compilations of the same code on the same cluster did not result in this error.  Any help is appreciated.
I used the config option: MPI_DIR = BUILD
-DG
Sent from my iPhone
_______________________________________________
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users
Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
_______________________________________________
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .