[Users] Problem compiling the Einstein Toolkit on Kraken

[Barry Wardell]

Hi Erik,

Thanks for your suggestions.

On 25/05/2010 19:42, Erik Schnetter wrote:
> Yes, this is a compiler problem.  The usual remedies are (in this order):
> 0. Delete configuration, build again
> 1. use a different compiler version
> 2. use a different compiler
> 3. change optimisation flags
> 4. blindly rearrange source code a bit to see whether this changes 
> anything
> 5. remove thorn from thorn list
> 6. use a different machine
>
> Round 0: Did you try this?  Sometimes there is a transmission error in 
> a file, or some other transient error that goes away if you try again.
Yes, the result is the same after deleting the configuration and 
building again
> Round 1: You could try using PGI 10.3, but when I tried this, I found 
> more compiler problems than with 9.0.4.  I'm attaching my option list 
> for 10.3 anyway.  I don't expect PGI to be interested in bug reports 
> against 9.0.4 that can't be reproduced with 10.3 any more.  PGI 10.4 
> should be available this month -- apparently PGI/Cray have a monthly 
> release cycle for compiler versions.
I see what you mean. I've just tried this with your option list and the 
build fails even sooner:

Preprocessing 
/nics/a/proj/cactus/barrywar/xt5/Cactus/arrangements/Carpet/CarpetLib/src/data.cc
Compiling 
/nics/a/proj/cactus/barrywar/xt5/Cactus/arrangements/Carpet/CarpetLib/src/data.cc
"/nics/a/proj/cactus/barrywar/xt5/Cactus/arrangements/Carpet/CarpetLib/src/cycle.h", 
line 243: warning:
           missing return statement at end of non-void function "getticks"
   }
   ^

pgCC-Fatal-/opt/pgi/10.3.0/linux86-64/10.3/bin/pgcpp1 TERMINATED by 
signal 11
Arguments to /opt/pgi/10.3.0/linux86-64/10.3/bin/pgcpp1
/opt/pgi/10.3.0/linux86-64/10.3/bin/pgcpp1 --llalign -Dunix -D__unix 
-D__unix__ -Dlinux -D__linux -D__linux__ -D__NO_MATH_INLINES 
-D__x86_64__ -D__LONG_MAX__=9223372036854775807L 
'-D__SIZE_TYPE__=unsigned long int' '-D__PTRDIFF_TYPE__=long int' 
-D__THROW= -D__extension__= -D__amd64__ -D__SSE__ -D__MMX__ -D__SSE2__ 
-D__SSE3__ -D__PGI -I/opt/cray/hdf5/1.8.4.1/hdf5-pgi/include 
-I/opt/mpt/3.5.1/xt/mpich2-pgi/include 
-I/opt/cray/hdf5/1.8.4.1/hdf5-pgi/include 
-I/opt/mpt/3.5.1/xt/mpich2-pgi/include 
-I/nics/a/proj/cactus/barrywar/xt5/Cactus/arrangements/Carpet/CarpetLib/src 
-I/nics/a/proj/cactus/barrywar/xt5/Cactus/arrangements/Carpet/CarpetLib/src/include 
-I/nics/a/proj/cactus/barrywar/xt5/Cactus/configs/ET/config-data 
-I/nics/a/proj/cactus/barrywar/xt5/Cactus/configs/ET/bindings/include 
-I/nics/a/proj/cactus/barrywar/xt5/Cactus/src/include 
-I/nics/a/proj/cactus/barrywar/xt5/Cactus/arrangements 
-I/nics/a/proj/cactus/barrywar/xt5/Cactus/configs/ET/bindings/Configuration/Thorns 
-I/nics/a/proj/cactus/barrywar/xt5/Cactus/arrangements/Carpet/CarpetLib/src 
-I/nics/a/proj/cactus/barrywar/xt5/Cactus/arrangements/Carpet/CarpetLib/src/include 
-DCRAY_XT -DMPICH_IGNORE_CXX_SEEK -DCRAY_XT -D_ISOC99_SOURCE 
-D_BSD_SOURCE -DTHORN_IS_CarpetLib -DCCODE 
-I/opt/pgi/10.3.0/linux86-64/10.3/include/CC 
-I/opt/pgi/10.3.0/linux86-64/10.3/include -I/usr/local/include 
-I/usr/lib64/gcc/x86_64-suse-linux/4.1.2/include 
-I/usr/lib64/gcc/x86_64-suse-linux/4.1.2/include -I/usr/include 
--gnu_version=40102 -g --dwarf2 --exceptions --mp -D_OPENMP=200805 
--exceptions -q -o /tmp/pgCCN7Fg1-WqoYPV.il 
/nics/a/proj/cactus/barrywar/xt5/Cactus/configs/ET/build/CarpetLib/data.cc
make[3]: *** [data.cc.o] Error 127
make[2]: *** [make.checked] Error 2

> Round 2: GCC on Kraken creates much slower code than PGI.  I'm not too 
> fond of the PathScale compilers.
I think PGI is the best option.
> Round 3: I was fighting with PGI optimisation options yesterday, and 
> my new preferred optimisation options are now
>
>     -O3 -Munroll=c:1 -Mscalarsse -Mcache_align -Mflushz
>
> which leave out some of the options implied by -fastsse as well as the 
> -Mvect=assoc.  Do you want to try these?
This seems to fix the problem. I changed the optimisation options in the 
current simfactory optionlist (using PGI 9.0.4) and no longer get the 
problem compiling AHFinder. What are the implications of using these new 
options instead of the existing ones?
> 4. blindly rearrange source code a bit to see whether this changes 
> anything
I'm glad I didn't have to do this.
> 5. remove thorn from thorn list
This is a possibility since I don't think I actually need the thorn. 
This thorn is in the ET though, so I guess someone else might encounter 
this problem at some stage.
> 6. use a different machine
Yes, it works fine with QueenBee and Damiana.

Barry