Good morning, i am trying to compile the developer version of the Einstein Toolkit on the Fermi machine (a Blue Gene/Q machine of Cineca, Italy). The compilation hangs when trying to compile GRHydro_WENOReconstruct.cc and after a while is aborted with almost no error informations:
#################################################################################### COMPILING arrangements/EinsteinEvolve/GRHydro/src/GRHydro_WENOReconstruct.cc "/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus/src/include/CommOverloadables.h", line 87.1: 1540-0081 (W) The attribute "noreturn" is not a valid type attribute. The attribute is ignored. "/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus/src/include/CommOverloadables.h", line 88.1: 1540-0081 (W) The attribute "noreturn" is not a valid type attribute. The attribute is ignored. /opt/ibmcmp/vacpp/bg/12.1/bin/.orig/bgxlC_r: 1501-224 (S) fatal error in /opt/ibmcmp/vac/bg/12.1/exe/ipa: signal 9 received make[3]: *** [GRHydro_WENOReconstruct.cc.o] Error 251 make[3]: Leaving directory `/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus/configs/sim/build/GRHydro' make[2]: *** [make.checked] Error 2 make[2]: Leaving directory `/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus/configs/sim/build/GRHydro' make[1]: *** [/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus/configs/sim/lib/libthorn_GRHydro.a] Error 2 make[1]: Leaving directory `/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus' make: *** [sim] Error 2 ###################################################################################
This error is also present in ET_2014_05 and ET_2014_11 stable versions. I am using the IBM compiler for Blue Gene/Q bgxlC_r version 12.1. In the configuration file under simfactory/mdb/optionlist i have set the following flags for cxx programs:
CXXFLAGS = -g -qarch=qp -qtune=qp -qrtti -qsuppress=1500-036:1506-1108 -qnokeyword=restrict -qalias=norestrict CXX_OPTIMISE_FLAGS = -O2 -qmaxmem=-1 -qhot -qlibmpi (the error is present also with -O0) CXX_OPENMP_FLAGS = -qsmp=omp:noauto
Have any of you encountered a similar error, or knows how to fix it? Thank you for your help,
Francesco Maione
__________________________________________ Francesco Maione Ph.D. student in Physics, Università di Parma Dipartimento di Fisica e Scienze della Terra Via G.P.Usberti 7/A I-43100 PARMA --- ITALY Tel: +39 (0521) 905218
Hi Francesco
On Thu, Mar 12, 2015 at 04:12:06PM +0100, Francesco Maione wrote:
Good morning, i am trying to compile the developer version of the Einstein Toolkit on the Fermi machine (a Blue Gene/Q machine of Cineca, Italy). The compilation hangs when trying to compile GRHydro_WENOReconstruct.cc and after a while is aborted with almost no error informations:
/opt/ibmcmp/vacpp/bg/12.1/bin/.orig/bgxlC_r: 1501-224 (S) fatal error in /opt/ibmcmp/vac/bg/12.1/exe/ipa: signal 9 received make[3]: *** [GRHydro_WENOReconstruct.cc.o] Error 251
Error 251 (according to [1], but probably also for C++) is "signal received"), which fits the line above. Signal 9 is a hard kill. This, and your statement "a while" suggests that the compiler was killed - likely by a script on the cluster head node that kills all processes that take "too long". Computing centers sometimes do this to prevent users from running compute jobs on the head node. Unfortunately, sometimes compiling can also take a long time, and scripts usually don't differentiate. Could this be the case? How long was "a while"? Would it work to compile inside a compute job - at least to test that theory?
Frank
[1] http://www-01.ibm.com/support/knowledgecenter/SS3KZ4_9.0.0/com.ibm.xlf111.bg...
Francesco
The error "signal 9" points either to an error in the compiler itself, or to resource exhaustion, e.g. the compiler is running out of memory.
There is probably no good way to remedy this. You can try: - compiling fewer files at the same time (make -j1) - reducing optimization settings (not just -O0, also e.g. removing -qhot) - choosing a different -qmaxmem value, e.g. 1 GByte (1000000000), or less if necessary
If this doesn't help, then you may have to change the source file, e.g. breaking it into several pieces that are "easier" on the compiler. You may also have to break large function in these pieces into several smaller functions, allowing you to generate even more pieces.
A very different approach would be to use a different compiler, e.g. gfortran.
Or you may be able to use run-time parameters that avoid calling any routine in this problematic file, so that you don't need to compile it at all.
In any way I would involve the system administrators. Maybe there is a newer (or older) version of the compiler you can try.
-erik
On Mar 12, 2015, at 11:12 , Francesco Maione francesco.maione@fis.unipr.it wrote:
Good morning, i am trying to compile the developer version of the Einstein Toolkit on the Fermi machine (a Blue Gene/Q machine of Cineca, Italy). The compilation hangs when trying to compile GRHydro_WENOReconstruct.cc and after a while is aborted with almost no error informations:
#################################################################################### COMPILING arrangements/EinsteinEvolve/GRHydro/src/GRHydro_WENOReconstruct.cc "/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus/src/include/CommOverloadables.h", line 87.1: 1540-0081 (W) The attribute "noreturn" is not a valid type attribute. The attribute is ignored. "/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus/src/include/CommOverloadables.h", line 88.1: 1540-0081 (W) The attribute "noreturn" is not a valid type attribute. The attribute is ignored. /opt/ibmcmp/vacpp/bg/12.1/bin/.orig/bgxlC_r: 1501-224 (S) fatal error in /opt/ibmcmp/vac/bg/12.1/exe/ipa: signal 9 received make[3]: *** [GRHydro_WENOReconstruct.cc.o] Error 251 make[3]: Leaving directory `/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus/configs/sim/build/GRHydro' make[2]: *** [make.checked] Error 2 make[2]: Leaving directory `/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus/configs/sim/build/GRHydro' make[1]: *** [/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus/configs/sim/lib/libthorn_GRHydro.a] Error 2 make[1]: Leaving directory `/gpfs/work/INF14_teongrav/francesco.maione/ET/dev/Cactus' make: *** [sim] Error 2 ###################################################################################
This error is also present in ET_2014_05 and ET_2014_11 stable versions. I am using the IBM compiler for Blue Gene/Q bgxlC_r version 12.1. In the configuration file under simfactory/mdb/optionlist i have set the following flags for cxx programs:
CXXFLAGS = -g -qarch=qp -qtune=qp -qrtti -qsuppress=1500-036:1506-1108 -qnokeyword=restrict -qalias=norestrict CXX_OPTIMISE_FLAGS = -O2 -qmaxmem=-1 -qhot -qlibmpi (the error is present also with -O0) CXX_OPENMP_FLAGS = -qsmp=omp:noauto
Have any of you encountered a similar error, or knows how to fix it? Thank you for your help,
Francesco Maione
Francesco Maione Ph.D. student in Physics, Università di Parma Dipartimento di Fisica e Scienze della Terra Via G.P.Usberti 7/A I-43100 PARMA --- ITALY Tel: +39 (0521) 905218
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
-- Erik Schnetter schnetter@gmail.com http://www.perimeterinstitute.ca/personal/eschnetter/
My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from https://sks-keyservers.net.
Hello all,
If this doesn't help, then you may have to change the source file, e.g. breaking it into several pieces that are "easier" on the compiler. You may also have to break large function in these pieces into several smaller functions, allowing you to generate even more pieces.
This would be a good thing to do in any case. Those routines in GRHydro instantiate many C++ templates for the different cases. Overall compilation speed probably improves if one makes one tiny C++ file per instantiation. The instantiation happens in these lines:
INSTANTIATE_RECONSTRUCTION_OPERATOR(GRHydro_WENOReconstruct1d_cxx00) INSTANTIATE_RECONSTRUCTION_OPERATOR(GRHydro_WENOReconstruct1d_cxx01) INSTANTIATE_RECONSTRUCTION_OPERATOR(GRHydro_WENOReconstruct1d_cxx10) INSTANTIATE_RECONSTRUCTION_OPERATOR(GRHydro_WENOReconstruct1d_cxx11)
*each* of which (in the macro INSTANTIATE_RECONSTRUCTION_OPERATOR) instantiates 12 copies of the given routine. The macro is defined in GRHydro_Reconstruct_drv_cxx.hh .
Yours, Roland
Hi Francesco,
On Mar 12, 2015, at 8:12 AM, Francesco Maione francesco.maione@fis.unipr.it wrote:
Have any of you encountered a similar error, or knows how to fix it? Thank you for your help,
I had similar issues compiling my Cactus based code (WhiskyTHC) on Fermi. The only solution I could find was to switch from IBM to gcc.
Best, David
users@lists.einsteintoolkit.org
-
David Radice -
Erik Schnetter -
Francesco Maione -
Frank Loeffler -
Roland Haas