here is the error messages that occur when running on the local cluster (compiled with intel compilers):<div><br></div><div><div>WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable &#39;HYDROBASE::rho&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable &#39;HYDROBASE::rho&#39;</div><div>WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable &#39;HYDROBASE::eps&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable &#39;HYDROBASE::eps&#39;</div><div>WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable &#39;HYDROBASE::press&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable &#39;HYDROBASE::press&#39;</div><div>WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[0]&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[0]&#39;</div><div>WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[1]&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[1]&#39;</div><div>WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[2]&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[2]&#39;</div><div>WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable &#39;HYDROBASE::w_lorentz&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable &#39;HYDROBASE::w_lorentz&#39;</div><div>WARNING[L1,P0] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::dens&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::dens&#39;</div><div>WARNING[L1,P0] (NaNChecker): There were 20094 NaN/Inf value(s) found in variable &#39;GRHYDRO::tau&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 20782 NaN/Inf value(s) found in variable &#39;GRHYDRO::tau&#39;</div><div>WARNING[L1,P0] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[0]&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[0]&#39;</div><div>WARNING[L1,P0] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[1]&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[1]&#39;</div><div>WARNING[L1,P0] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[2]&#39;</div>
<div>WARNING[L1,P1] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[2]&#39;</div><div>WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable &#39;HYDROBASE::rho&#39;</div>
<div>WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable &#39;HYDROBASE::rho&#39;</div><div>WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable &#39;HYDROBASE::rho&#39;</div>
<div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::rho&#39;</div><div>WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable &#39;HYDROBASE::eps&#39;</div>
<div>WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable &#39;HYDROBASE::eps&#39;</div><div>WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable &#39;HYDROBASE::eps&#39;</div>
<div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::eps&#39;</div><div>WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable &#39;HYDROBASE::press&#39;</div>
<div>WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable &#39;HYDROBASE::press&#39;</div><div>WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable &#39;HYDROBASE::press&#39;</div>
<div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::press&#39;</div><div>WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable &#39;HYDROBASE::rho&#39;</div>
<div>WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[0]&#39;</div><div>WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[0]&#39;</div>
<div>WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[0]&#39;</div><div>WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable &#39;HYDROBASE::eps&#39;</div>
<div>WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[1]&#39;</div><div>WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[1]&#39;</div>
<div>WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[1]&#39;</div><div>WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable &#39;HYDROBASE::press&#39;</div>
<div>WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[2]&#39;</div><div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[0]&#39;</div>
<div>WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[2]&#39;</div><div>WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[2]&#39;</div>
<div>WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[0]&#39;</div><div>WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable &#39;HYDROBASE::w_lorentz&#39;</div>
<div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[1]&#39;</div><div>WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable &#39;HYDROBASE::w_lorentz&#39;</div>
<div>WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[1]&#39;</div><div>WARNING[L1,P7] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::dens&#39;</div>
<div>WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable &#39;HYDROBASE::w_lorentz&#39;</div><div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[2]&#39;</div>
<div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::rho&#39;</div><div>WARNING[L1,P3] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::dens&#39;</div>
<div>WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[2]&#39;</div><div>WARNING[L1,P7] (NaNChecker): There were 1261 NaN/Inf value(s) found in variable &#39;GRHYDRO::tau&#39;</div>
<div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::w_lorentz&#39;</div><div>WARNING[L1,P6] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::dens&#39;</div>
<div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::eps&#39;</div><div>WARNING[L1,P3] (NaNChecker): There were 33853 NaN/Inf value(s) found in variable &#39;GRHYDRO::tau&#39;</div>
<div>WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable &#39;HYDROBASE::w_lorentz&#39;</div><div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::dens&#39;</div>
<div>WARNING[L1,P6] (NaNChecker): There were 1179 NaN/Inf value(s) found in variable &#39;GRHYDRO::tau&#39;</div><div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::press&#39;</div>
<div>WARNING[L1,P3] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[0]&#39;</div><div>WARNING[L1,P2] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::dens&#39;</div>
<div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::tau&#39;</div><div>WARNING[L1,P3] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[1]&#39;</div>
<div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[0]&#39;</div><div>WARNING[L1,P7] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[0]&#39;</div>
<div>WARNING[L1,P2] (NaNChecker): There were 33396 NaN/Inf value(s) found in variable &#39;GRHYDRO::tau&#39;</div><div>WARNING[L1,P6] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[0]&#39;</div>
<div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[0]&#39;</div><div>WARNING[L1,P3] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[2]&#39;</div>
<div>WARNING[L1,P7] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[1]&#39;</div><div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[1]&#39;</div>
<div>WARNING[L1,P2] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[0]&#39;</div><div>WARNING[L1,P6] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[1]&#39;</div>
<div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[1]&#39;</div><div>WARNING[L1,P7] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[2]&#39;</div>
<div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::vel[2]&#39;</div><div>WARNING[L1,P2] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[1]&#39;</div>
<div>WARNING[L1,P6] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[2]&#39;</div><div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;HYDROBASE::w_lorentz&#39;</div>
<div>WARNING[L1,P5] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[2]&#39;</div><div>WARNING[L1,P2] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[2]&#39;</div>
<div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::dens&#39;</div><div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::tau&#39;</div>
<div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[0]&#39;</div><div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[1]&#39;</div>
<div>WARNING[L1,P4] (NaNChecker): There were 1 NaN/Inf value(s) found in variable &#39;GRHYDRO::scon[2]&#39;</div><div>MPI: On host vives, Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 7, Process 390844 received signal SIGFPE(8)</div>
<div><br></div><div><br></div><div>MPI: --------stack traceback-------</div><div>MPI: </div><div>MPI: A problem internal to GDB has been detected,</div><div>MPI: further debugging may prove unreliable.</div><div>MPI: Quit this debugging session? (y or n) [answered Y; input not from terminal]</div>
<div>MPI: ../../gdb/dwarf2read.c:9825: internal-error: could not find partial DIE 0x5839d3e in cache [from module /proc/390844/exe]</div><div>MPI: </div><div>MPI: A problem internal to GDB has been detected,</div><div>MPI: further debugging may prove unreliable.</div>
<div>MPI: Create a core file of GDB? (y or n) [answered Y; input not from terminal]</div><div><br></div><div>MPI: -----stack traceback ends-----</div><div>MPI: On host vives, Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 7, Process 390844: Dumping core on signal SIGFPE(8) into directory /usus/va/vasme/cactus/Cactus</div>
<div>MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 2, Process 390839: Core dump on signal SIGFPE(8) suppressed.</div><div>MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 0, Process 390825: Core dump on signal SIGFPE(8) suppressed.</div>
<div>MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 1, Process 390826: Core dump on signal SIGFPE(8) suppressed.</div><div>MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 6, Process 390843: Core dump on signal SIGFPE(8) suppressed.</div>
<div>MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 3, Process 390840: Core dump on signal SIGFPE(8) suppressed.</div><div>MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 5, Process 390842: Core dump on signal SIGFPE(8) suppressed.</div>
<div>MPI: Program /usus/va/vasme/cactus/Cactus/exe/cactus_ET_Tori_ID, Rank 4, Process 390841: Core dump on signal SIGFPE(8) suppressed.</div><div>MPI: MPI_COMM_WORLD rank 1 has terminated without calling MPI_Finalize()</div>
<div>MPI: aborting job</div><div>MPI: Received signal 9</div><div><br></div><div><br></div><div>i also made a run with <i>export OMP_NUM_THREADS=1 </i>, showing the same errors as before...</div><div><br></div><div>the compiler options from the optionlist are:</div>
<div><br></div><div>imac:</div><div><br></div><div><div>CPP = /usr/local/bin/cpp</div><div>FPP = /usr/local/bin/cpp</div><div>CC  = /Users/vass/programs/opt/openMPI-gcc4.7/bin/mpicc</div><div>CXX = /Users/vass/programs/opt/openMPI-gcc4.7/bin/mpicxx</div>
<div>F77 = /Users/vass/programs/opt/openMPI-gcc4.7/bin/mpif77</div><div>F90 = /Users/vass/programs/opt/openMPI-gcc4.7/bin/mpif90</div><div><br></div><div>AR      = ar </div><div>ARFLAGS = ruv </div><div><br></div><div># -fmudflapth does not work with gcc 4.2.0</div>
<div># -march=native prevents undefined references to ___sync_fetch_and_add_4</div><div># -malign-double leads to crashes in Fortran I/O</div><div>CPPFLAGS = -fPIC #-DOMPI_SKIP_MPICXX -DTAU_VERSION_NUMBER=21803</div><div>
FPPFLAGS = -traditional</div><div>CFLAGS   = -march=corei7 -mtune=corei7 -pipe -g3 -m128bit-long-double  -std=gnu99</div><div>CXXFLAGS = -march=corei7 -mtune=corei7 -pipe -g3  -O2 -DNDEBUG -pedantic -fpermissive #-Wall -W -Wundef -Wshadow -Wcast-qual -Wcast-align -Wconversion -Winline -Wabi -Wold-style-cast -Woverloaded-virtual</div>
<div>F77FLAGS = -march=corei7 -mtune=corei7 -pipe -g3 -m128bit-long-double  -fcray-pointer -ffixed-line-length-none</div><div>F90FLAGS = -march=corei7 -mtune=corei7 -pipe -g3 -m128bit-long-double  -fcray-pointer -ffixed-line-length-none</div>
<div><br></div><div>LDFLAGS = -framework Accelerate -lgfortran</div><div><br></div><div>C_LINE_DIRECTIVES = yes</div><div>F_LINE_DIRECTIVES = yes</div><div><br></div><div>REAL16_KIND = 10</div><div><br></div><div>DEBUG           = no</div>
<div>CPP_DEBUG_FLAGS = -DCARPET_DEBUG</div><div>FPP_DEBUG_FLAGS = -DCARPET_DEBUG</div><div>C_DEBUG_FLAGS   = -fbounds-check -fstack-protector-all -ftrapv</div><div>CXX_DEBUG_FLAGS = -fbounds-check -fstack-protector-all -ftrapv</div>
<div>F77_DEBUG_FLAGS = -fcheck=bounds,do,mem,pointer,recursion -finit-character=65 -finit-integer=42424242 -finit-real=nan -fstack-protector-all -ftrapv -fwhole-file</div><div>F90_DEBUG_FLAGS = -fcheck=bounds,do,mem,pointer,recursion -finit-character=65 -finit-integer=42424242 -finit-real=nan -fstack-protector-all -ftrapv -fwhole-file</div>
<div><br></div><div># Changing ANSI C semantics:</div><div># -funsafe-loop-optimizations -ffast-math-errno</div><div># Graphite optimisations are not implemented:</div><div># -floop-interchange -floop-strip-mine -floop-block</div>
<div># LTO support is not enabled:</div><div># -flto</div><div>OPTIMISE           = yes</div><div>CPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG</div><div>FPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG</div><div>C_OPTIMISE_FLAGS   = -O3 -funsafe-loop-optimizations -fsee -ftree-loop-im -fivopts -fvect-cost-model -funroll-loops -funroll-all-loops -fvariable-expansion-in-unroller -fprefetch-loop-arrays -ffast-math -fassociative-math -freciprocal-math -fno-trapping-math -fexcess-precision=fast -ftree-vectorize</div>
<div>CXX_OPTIMISE_FLAGS = -O3 -funsafe-loop-optimizations -fsee -ftree-loop-im -fivopts -fvect-cost-model -funroll-loops -funroll-all-loops -fvariable-expansion-in-unroller -fprefetch-loop-arrays -ffast-math -fassociative-math -freciprocal-math -fno-trapping-math -fexcess-precision=fast -ftree-vectorize</div>
<div>F77_OPTIMISE_FLAGS = -O3 -funsafe-loop-optimizations -fsee -ftree-loop-im -fivopts -fvect-cost-model -funroll-loops -funroll-all-loops -fvariable-expansion-in-unroller -fprefetch-loop-arrays -ffast-math -fassociative-math -freciprocal-math -fno-trapping-math -fexcess-precision=fast -fno-protect-parens -ftree-vectorize</div>
<div>F90_OPTIMISE_FLAGS = -O3 -funsafe-loop-optimizations -fsee -ftree-loop-im -fivopts -fvect-cost-model -funroll-loops -funroll-all-loops -fvariable-expansion-in-unroller -fprefetch-loop-arrays -ffast-math -fassociative-math -freciprocal-math -fno-trapping-math -fexcess-precision=fast -fno-protect-parens -ftree-vectorize</div>
<div><br></div><div>PROFILE           = no</div><div>CPP_PROFILE_FLAGS =</div><div>FPP_PROFILE_FLAGS =</div><div>C_PROFILE_FLAGS   = -pg</div><div>CXX_PROFILE_FLAGS = -pg</div><div>F77_PROFILE_FLAGS = -pg</div><div>F90_PROFILE_FLAGS = -pg</div>
<div><br></div><div>OPENMP           = yes</div><div>CPP_OPENMP_FLAGS = -fopenmp</div><div>FPP_OPENMP_FLAGS = -fopenmp</div><div>C_OPENMP_FLAGS   = -fopenmp</div><div>CXX_OPENMP_FLAGS = -fopenmp</div><div>F77_OPENMP_FLAGS = -fopenmp</div>
<div>F90_OPENMP_FLAGS = -fopenmp</div><div><br></div><div># -Wuninitialized is not supported without -O</div><div>WARN           = no</div><div>#CPP_WARN_FLAGS = -Wall</div><div>#FPP_WARN_FLAGS = -Wall</div><div>#C_WARN_FLAGS   = -Wall -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-declarations -Wbad-function-cast -Wsign-compare</div>
<div>#CXX_WARN_FLAGS = -Wall -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Woverloaded-virtual</div><div>#F77_WARN_FLAGS = -Wall -Wshadow -Wsurprising</div><div>#F90_WARN_FLAGS = -Wall -Wshadow -Wsurprising</div></div>
<div><br></div><div>and local cluster:</div><div><br></div><div><div>CPP = cpp</div><div>FPP = cpp</div><div>CC  = icc</div><div>CXX = icpc</div><div>F77 = ifort</div><div>F90 = ifort</div><div>FPPFLAGS= -traditional</div>
<div><br></div><div>CPPFLAGS = -fPIC</div><div>FPPFLAGS = -traditional</div><div>CFLAGS   = -g -std=gnu99 -fPIC</div><div>CXXFLAGS = -g -fPIC</div><div>F77FLAGS = -g -traceback -fPIC </div><div>F90FLAGS = -g -traceback -fPIC</div>
<div><br></div><div>LIBDIRS = /util/software/Intel/INTEL_12.0/composerxe-2011.0.084/compiler/lib/intel64 /util/software/Intel/INTEL_12.0/composerxe-2011.0.084/compiler/lib/intel64</div><div><br></div><div>C_LINE_DIRECTIVE = yes</div>
<div>F_LINE_DIRECTIVE = yes</div><div><br></div><div># -O2 and -O3 leads to very long (infinite?) compile times</div><div>#OPTIMISE           = yes</div><div>#CPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG</div><div>#FPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG</div>
<div>#C_OPTIMISE_FLAGS   = -O -funroll-loops -ftz</div><div>#CXX_OPTIMISE_FLAGS = -O -funroll-loops -ftz</div><div>#F77_OPTIMISE_FLAGS = -O -funroll-loops -ftz</div><div>#F90_OPTIMISE_FLAGS = -O -funroll-loops -ftz</div><div>
<br></div><div>#PROFILE           = no</div><div>#CPP_PROFILE_FLAGS =</div><div>#FPP_PROFILE_FLAGS =</div><div>#C_PROFILE_FLAGS   = -p</div><div>#CXX_PROFILE_FLAGS = -p</div><div>#F77_PROFILE_FLAGS = -p</div><div>#F90_PROFILE_FLAGS = -p</div>
<div><br></div><div>OPENMP           = yes</div><div>CPP_OPENMP_FLAGS = -openmp</div><div>FPP_OPENMP_FLAGS = -fopenmp</div><div>C_OPENMP_FLAGS   = -openmp</div><div>CXX_OPENMP_FLAGS = -openmp</div><div>F77_OPENMP_FLAGS = -openmp</div>
<div>F90_OPENMP_FLAGS = -openmp</div><div><br></div><div><br></div><div>#WARN           = yes</div><div>#CPP_WARN_FLAGS = -Wall</div><div>#FPP_WARN_FLAGS = -Wall</div><div>#C_WARN_FLAGS   = -Wall -wd810 -wd981 -wd1419</div>
<div>#CXX_WARN_FLAGS = -Wall -wd810 -wd981 -wd1419</div><div>#F77_WARN_FLAGS = -warn all</div><div>#F90_WARN_FLAGS = -warn all</div></div><div><br></div><div>thanks alot,</div><div><br></div><div>best wishes,</div><div><br>
</div><div>Vassili</div><br><div class="gmail_quote">On Mon, Oct 8, 2012 at 8:39 AM, Roland Haas <span dir="ltr">&lt;<a href="mailto:roland.haas@physics.gatech.edu" target="_blank">roland.haas@physics.gatech.edu</a>&gt;</span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello Vasilios,<br>
<div class="im"><br>
&gt; i don&#39;t know whats causing this, could the problem be caused by the<br>
&gt; different compilers on the two machines?<br>
</div>We run the Cactus testsuites on all supported machine before releases.<br>
With very few exceptions for exotic machines/compilers all tests pass on<br>
all machines with all compilers (see<br>
<a href="http://einsteintoolkit.org/release-info/parse_testsuite_results.php" target="_blank">http://einsteintoolkit.org/release-info/parse_testsuite_results.php</a>)<br>
using identical test data. So what I am trying to say is that while<br>
there will be differences between different compilers (and compiler<br>
options), they tend to be small (1e-12 relative deviation or so).<br>
Exceptions are codes that discretely change behaviour based on data (ie.<br>
shock capturing) and everything that involves 1/r terms and evaluations<br>
close to zero. In particular Intel and gcc compiler will differ (unless<br>
both are passed the -O0 option) since Intel defaults to --fast-math<br>
which produces non ieee compliant floating point arithmetic, while<br>
gfortran if standards comforming unless to otherwise.<br>
<br>
My guess based on you saying &quot;it does not work&quot; (please provide more<br>
information: which machines, how many MPI processes and OpenMP threads<br>
each, ideally the full output of Cactus/Carpet is useful) would be to<br>
check that you run with the same number of MPI processes and OpenMP<br>
threads on both machines (Carpet outputs this information on startup).<br>
If the number of threads is 1 in one case and larger than one in the<br>
other then you have an OpenMP problem (somewhere).<br>
<br>
Yours,<br>
Roland<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
My email is as private as my paper mail. I therefore support encrypting<br>
and signing email messages. Get my PGP key from <a href="http://keys.gnupg.net" target="_blank">http://keys.gnupg.net</a>.<br>
<br>
</font></span><br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@einsteintoolkit.org">Users@einsteintoolkit.org</a><br>
<a href="http://lists.einsteintoolkit.org/mailman/listinfo/users" target="_blank">http://lists.einsteintoolkit.org/mailman/listinfo/users</a><br>
<br></blockquote></div><br></div>