Hello all,
i have a working installation of the einstein toolkit (compilation finished with all done.) on both my imac at work and on a local cluster at the university of valencia...both are installation of the 2012_05 release...
i have written thorns to read in and extrapolate RNSID-like initial data to the cartesian grid of the einstein toolkit..
the code runs on my imac, but gives NaNs and therefore segmentation faults on the cluster..
the imac version was compiled with g++-4.7 and gfortran-4.7 and the version on the cluster with ifort 12.0 (although i think that this shouldn't be an issue??)
the thornlist used to compile the two configurations is the one coming with a fresh checkout, and i have just added my two thorns to it....
running on the imac, the data is read in and interpolated without errors or nans popping up...
on the cluster, there seems to be a problem with the initial data, and the code crashes on segmentation faults
i don't know whats causing this, could the problem be caused by the different compilers on the two machines?
best wishes,
Vassili
Hello Vasilios,
i don't know whats causing this, could the problem be caused by the different compilers on the two machines?
We run the Cactus testsuites on all supported machine before releases. With very few exceptions for exotic machines/compilers all tests pass on all machines with all compilers (see http://einsteintoolkit.org/release-info/parse_testsuite_results.php) using identical test data. So what I am trying to say is that while there will be differences between different compilers (and compiler options), they tend to be small (1e-12 relative deviation or so). Exceptions are codes that discretely change behaviour based on data (ie. shock capturing) and everything that involves 1/r terms and evaluations close to zero. In particular Intel and gcc compiler will differ (unless both are passed the -O0 option) since Intel defaults to --fast-math which produces non ieee compliant floating point arithmetic, while gfortran if standards comforming unless to otherwise.
My guess based on you saying "it does not work" (please provide more information: which machines, how many MPI processes and OpenMP threads each, ideally the full output of Cactus/Carpet is useful) would be to check that you run with the same number of MPI processes and OpenMP threads on both machines (Carpet outputs this information on startup). If the number of threads is 1 in one case and larger than one in the other then you have an OpenMP problem (somewhere).
Yours, Roland
here is the error messages that occur when running on the local cluster (compiled with intel compilers):
WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable 'HYDROBASE::rho' WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable 'HYDROBASE::rho' WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable 'HYDROBASE::eps' WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable 'HYDROBASE::eps' WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable 'HYDROBASE::press' WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable 'HYDROBASE::press' WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable 'HYDROBASE::vel[0]' WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable 'HYDROBASE::vel[0]' WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable 'HYDROBASE::vel[1]' WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable 'HYDROBASE::vel[1]' WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable 'HYDROBASE::vel[2]' WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable 'HYDROBASE::vel[2]' WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable 'HYDROBASE::w_lorentz' WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable 'HYDROBASE::w_lorentz' WARNING[L1,P0] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::dens' WARNING[L1,P1] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::dens' WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable 'GRHYDRO::tau' WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable 'GRHYDRO::tau' WARNING[L1,P0] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[0]' WARNING[L1,P1] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[0]' WARNING[L1,P0] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[1]' WARNING[L1,P1] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[1]' WARNING[L1,P0] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[2]' WARNING[L1,P1] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[2]' WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable 'HYDROBASE::rho' WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable 'HYDROBASE::rho' WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable 'HYDROBASE::rho' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::rho' WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable 'HYDROBASE::eps' WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable 'HYDROBASE::eps' WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable 'HYDROBASE::eps' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::eps' WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable 'HYDROBASE::press' WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable 'HYDROBASE::press' WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable 'HYDROBASE::press' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::press' WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable 'HYDROBASE::rho' WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable 'HYDROBASE::vel[0]' WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable 'HYDROBASE::vel[0]' WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable 'HYDROBASE::vel[0]' WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable 'HYDROBASE::eps' WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable 'HYDROBASE::vel[1]' WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable 'HYDROBASE::vel[1]' WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable 'HYDROBASE::vel[1]' WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable 'HYDROBASE::press' WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable 'HYDROBASE::vel[2]' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::vel[0]' WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable 'HYDROBASE::vel[2]' WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable 'HYDROBASE::vel[2]' WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable 'HYDROBASE::vel[0]' WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable 'HYDROBASE::w_lorentz' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::vel[1]' WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable 'HYDROBASE::w_lorentz' WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable 'HYDROBASE::vel[1]' WARNING[L1,P7] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::dens' WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable 'HYDROBASE::w_lorentz' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::vel[2]' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::rho' WARNING[L1,P3] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::dens' WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable 'HYDROBASE::vel[2]' WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable 'GRHYDRO::tau' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::w_lorentz' WARNING[L1,P6] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::dens' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::eps' WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable 'GRHYDRO::tau' WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable 'HYDROBASE::w_lorentz' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::dens' WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable 'GRHYDRO::tau' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::press' WARNING[L1,P3] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[0]' WARNING[L1,P2] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::dens' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::tau' WARNING[L1,P3] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[1]' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::vel[0]' WARNING[L1,P7] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[0]' WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable 'GRHYDRO::tau' WARNING[L1,P6] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[0]' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[0]' WARNING[L1,P3] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[2]' WARNING[L1,P7] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[1]' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::vel[1]' WARNING[L1,P2] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[0]' WARNING[L1,P6] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[1]' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[1]' WARNING[L1,P7] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[2]' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::vel[2]' WARNING[L1,P2] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[1]' WARNING[L1,P6] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[2]' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'HYDROBASE::w_lorentz' WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[2]' WARNING[L1,P2] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[2]' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::dens' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::tau' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[0]' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[1]' WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable 'GRHYDRO::scon[2]' MPI: On host vives, Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 7, Process 390844 received signal SIGFPE(8)
MPI: --------stack traceback------- MPI: MPI: A problem internal to GDB has been detected, MPI: further debugging may prove unreliable. MPI: Quit this debugging session? (y or n) [answered Y; input not from terminal] MPI: ../../gdb/dwarf2read.c:9825: internal-error: could not find partial DIE 0x5839d3e in cache [from module /proc/390844/exe] MPI: MPI: A problem internal to GDB has been detected, MPI: further debugging may prove unreliable. MPI: Create a core file of GDB? (y or n) [answered Y; input not from terminal]
MPI: -----stack traceback ends----- MPI: On host vives, Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 7, Process 390844: Dumping core on signal SIGFPE(8) into directory /usus/va/vasme/cactus/Cactus MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 2, Process 390839: Core dump on signal SIGFPE(8) suppressed. MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 0, Process 390825: Core dump on signal SIGFPE(8) suppressed. MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 1, Process 390826: Core dump on signal SIGFPE(8) suppressed. MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 6, Process 390843: Core dump on signal SIGFPE(8) suppressed. MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 3, Process 390840: Core dump on signal SIGFPE(8) suppressed. MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 5, Process 390842: Core dump on signal SIGFPE(8) suppressed. MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 4, Process 390841: Core dump on signal SIGFPE(8) suppressed. MPI: MPI_COMM_WORLD rank 1 has terminated without calling MPI_Finalize() MPI: aborting job MPI: Received signal 9
i also made a run with *export OMP_NUM_THREADS=1 *, showing the same errors as before...
the compiler options from the optionlist are:
imac:
CPP = /usr/local/bin/cpp FPP = /usr/local/bin/cpp CC = /Users/vass/programs/opt/openMPI-gcc4.7/bin/mpicc CXX = /Users/vass/programs/opt/openMPI-gcc4.7/bin/mpicxx F77 = /Users/vass/programs/opt/openMPI-gcc4.7/bin/mpif77 F90 = /Users/vass/programs/opt/openMPI-gcc4.7/bin/mpif90
AR = ar ARFLAGS = ruv
# -fmudflapth does not work with gcc 4.2.0 # -march=native prevents undefined references to ___sync_fetch_and_add_4 # -malign-double leads to crashes in Fortran I/O CPPFLAGS = -fPIC #-DOMPI_SKIP_MPICXX -DTAU_VERSION_NUMBER=21803 FPPFLAGS = -traditional CFLAGS = -march=corei7 -mtune=corei7 -pipe -g3 -m128bit-long-double -std=gnu99 CXXFLAGS = -march=corei7 -mtune=corei7 -pipe -g3 -O2 -DNDEBUG -pedantic -fpermissive #-Wall -W -Wundef -Wshadow -Wcast-qual -Wcast-align -Wconversion -Winline -Wabi -Wold-style-cast -Woverloaded-virtual F77FLAGS = -march=corei7 -mtune=corei7 -pipe -g3 -m128bit-long-double -fcray-pointer -ffixed-line-length-none F90FLAGS = -march=corei7 -mtune=corei7 -pipe -g3 -m128bit-long-double -fcray-pointer -ffixed-line-length-none
LDFLAGS = -framework Accelerate -lgfortran
C_LINE_DIRECTIVES = yes F_LINE_DIRECTIVES = yes
REAL16_KIND = 10
DEBUG = no CPP_DEBUG_FLAGS = -DCARPET_DEBUG FPP_DEBUG_FLAGS = -DCARPET_DEBUG C_DEBUG_FLAGS = -fbounds-check -fstack-protector-all -ftrapv CXX_DEBUG_FLAGS = -fbounds-check -fstack-protector-all -ftrapv F77_DEBUG_FLAGS = -fcheck=bounds,do,mem,pointer,recursion -finit-character=65 -finit-integer=42424242 -finit-real=nan -fstack-protector-all -ftrapv -fwhole-file F90_DEBUG_FLAGS = -fcheck=bounds,do,mem,pointer,recursion -finit-character=65 -finit-integer=42424242 -finit-real=nan -fstack-protector-all -ftrapv -fwhole-file
# Changing ANSI C semantics: # -funsafe-loop-optimizations -ffast-math-errno # Graphite optimisations are not implemented: # -floop-interchange -floop-strip-mine -floop-block # LTO support is not enabled: # -flto OPTIMISE = yes CPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG FPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG C_OPTIMISE_FLAGS = -O3 -funsafe-loop-optimizations -fsee -ftree-loop-im -fivopts -fvect-cost-model -funroll-loops -funroll-all-loops -fvariable-expansion-in-unroller -fprefetch-loop-arrays -ffast-math -fassociative-math -freciprocal-math -fno-trapping-math -fexcess-precision=fast -ftree-vectorize CXX_OPTIMISE_FLAGS = -O3 -funsafe-loop-optimizations -fsee -ftree-loop-im -fivopts -fvect-cost-model -funroll-loops -funroll-all-loops -fvariable-expansion-in-unroller -fprefetch-loop-arrays -ffast-math -fassociative-math -freciprocal-math -fno-trapping-math -fexcess-precision=fast -ftree-vectorize F77_OPTIMISE_FLAGS = -O3 -funsafe-loop-optimizations -fsee -ftree-loop-im -fivopts -fvect-cost-model -funroll-loops -funroll-all-loops -fvariable-expansion-in-unroller -fprefetch-loop-arrays -ffast-math -fassociative-math -freciprocal-math -fno-trapping-math -fexcess-precision=fast -fno-protect-parens -ftree-vectorize F90_OPTIMISE_FLAGS = -O3 -funsafe-loop-optimizations -fsee -ftree-loop-im -fivopts -fvect-cost-model -funroll-loops -funroll-all-loops -fvariable-expansion-in-unroller -fprefetch-loop-arrays -ffast-math -fassociative-math -freciprocal-math -fno-trapping-math -fexcess-precision=fast -fno-protect-parens -ftree-vectorize
PROFILE = no CPP_PROFILE_FLAGS = FPP_PROFILE_FLAGS = C_PROFILE_FLAGS = -pg CXX_PROFILE_FLAGS = -pg F77_PROFILE_FLAGS = -pg F90_PROFILE_FLAGS = -pg
OPENMP = yes CPP_OPENMP_FLAGS = -fopenmp FPP_OPENMP_FLAGS = -fopenmp C_OPENMP_FLAGS = -fopenmp CXX_OPENMP_FLAGS = -fopenmp F77_OPENMP_FLAGS = -fopenmp F90_OPENMP_FLAGS = -fopenmp
# -Wuninitialized is not supported without -O WARN = no #CPP_WARN_FLAGS = -Wall #FPP_WARN_FLAGS = -Wall #C_WARN_FLAGS = -Wall -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-declarations -Wbad-function-cast -Wsign-compare #CXX_WARN_FLAGS = -Wall -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Woverloaded-virtual #F77_WARN_FLAGS = -Wall -Wshadow -Wsurprising #F90_WARN_FLAGS = -Wall -Wshadow -Wsurprising
and local cluster:
CPP = cpp FPP = cpp CC = icc CXX = icpc F77 = ifort F90 = ifort FPPFLAGS= -traditional
CPPFLAGS = -fPIC FPPFLAGS = -traditional CFLAGS = -g -std=gnu99 -fPIC CXXFLAGS = -g -fPIC F77FLAGS = -g -traceback -fPIC F90FLAGS = -g -traceback -fPIC
LIBDIRS = /util/software/Intel/INTEL_12.0/composerxe-2011.0.084/compiler/lib/intel64 /util/software/Intel/INTEL_12.0/composerxe-2011.0.084/compiler/lib/intel64
C_LINE_DIRECTIVE = yes F_LINE_DIRECTIVE = yes
# -O2 and -O3 leads to very long (infinite?) compile times #OPTIMISE = yes #CPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG #FPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG #C_OPTIMISE_FLAGS = -O -funroll-loops -ftz #CXX_OPTIMISE_FLAGS = -O -funroll-loops -ftz #F77_OPTIMISE_FLAGS = -O -funroll-loops -ftz #F90_OPTIMISE_FLAGS = -O -funroll-loops -ftz
#PROFILE = no #CPP_PROFILE_FLAGS = #FPP_PROFILE_FLAGS = #C_PROFILE_FLAGS = -p #CXX_PROFILE_FLAGS = -p #F77_PROFILE_FLAGS = -p #F90_PROFILE_FLAGS = -p
OPENMP = yes CPP_OPENMP_FLAGS = -openmp FPP_OPENMP_FLAGS = -fopenmp C_OPENMP_FLAGS = -openmp CXX_OPENMP_FLAGS = -openmp F77_OPENMP_FLAGS = -openmp F90_OPENMP_FLAGS = -openmp
#WARN = yes #CPP_WARN_FLAGS = -Wall #FPP_WARN_FLAGS = -Wall #C_WARN_FLAGS = -Wall -wd810 -wd981 -wd1419 #CXX_WARN_FLAGS = -Wall -wd810 -wd981 -wd1419 #F77_WARN_FLAGS = -warn all #F90_WARN_FLAGS = -warn all
thanks alot,
best wishes,
Vassili
On Mon, Oct 8, 2012 at 8:39 AM, Roland Haas roland.haas@physics.gatech.eduwrote:
Hello Vasilios,
i don't know whats causing this, could the problem be caused by the different compilers on the two machines?
We run the Cactus testsuites on all supported machine before releases. With very few exceptions for exotic machines/compilers all tests pass on all machines with all compilers (see http://einsteintoolkit.org/release-info/parse_testsuite_results.php) using identical test data. So what I am trying to say is that while there will be differences between different compilers (and compiler options), they tend to be small (1e-12 relative deviation or so). Exceptions are codes that discretely change behaviour based on data (ie. shock capturing) and everything that involves 1/r terms and evaluations close to zero. In particular Intel and gcc compiler will differ (unless both are passed the -O0 option) since Intel defaults to --fast-math which produces non ieee compliant floating point arithmetic, while gfortran if standards comforming unless to otherwise.
My guess based on you saying "it does not work" (please provide more information: which machines, how many MPI processes and OpenMP threads each, ideally the full output of Cactus/Carpet is useful) would be to check that you run with the same number of MPI processes and OpenMP threads on both machines (Carpet outputs this information on startup). If the number of threads is 1 in one case and larger than one in the other then you have an OpenMP problem (somewhere).
Yours, Roland
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
users@lists.einsteintoolkit.org
-
Roland Haas -
Vassilios Mewes