Hello Roland,
Thanks for the comments on the issues. I have attached the config.log file for Part A, and I'll see what can be done about Part B.
For Part C, I found a LAPACK library in our HPC, and the compilation process was completed without that warning. The Helloworld is also executed correctly.
Currently, with this ET, I am facing issue on executing the gallery BNSM simulation on 2 nodes. The info in the command line goes as follows:
*./simfactory/bin/sim create-submit nsns_p32_t24_8 --procs=32 --ppn=16 --num-threads=1 --ppn-used=16 --num-smt=1 --parfile=par/nsnstohmns1.par --walltime=25:00:00*
In the out file, the error is:
*+ mpiexec -n 32 -npernode 16 /home2/mallick/simulations2/nsns_p32_t24_11/SIMFACTORY/exe/cactus_sim -L 3 /home2/mallick/simulations2/nsns_p32_t24_11/output-0000/nsnstohmns1.par--------------------------------------------------------------------------Your job has requested more processes than the ppr forthis topology can support: App: /home2/mallick/simulations2/nsns_p32_t24_11/SIMFACTORY/exe/cactus_sim Number of procs: 32 PPR: 16:nodePlease revise the conflict and try again.--------------------------------------------------------------------------Simfactory Done at date: Mon 14 Nov 2022 07:11:57 PM IST*
The simulations seem to work fine with 1 node, that is, --procs set to 16,8 or 4 keeping --ppn to 16. I assume the mpiexec command in runscript needs to be updated to provide clarity to the mpi library while using more than 1 node.
I need some help here, I don't know how to proceed at this point. I am attaching the run, ini, cfg, sub script used for this ET and the output file for reference. Thanks in advance for all the help.
Regards Shamim Haque Senior Research Fellow (SRF) Department of Physics IISER Bhopal
ᐧ
On Mon, Nov 14, 2022 at 8:22 PM Roland Haas rhaas@illinois.edu wrote:
Hello Shamim Haque,
*Part A:* Keeping compiler Intel 2013, I tried to add lines -gcc_name and -gxx_name to CFLAGS and CXXFLAGS pointing to gcc/7.3.0. The error while compiling
is
the following:
*checking for C++ lambda expressions... yeschecking for C++ range-based
for
statements... noCactus requires a C++11 compiler -- check your C++
compiler
and C++ compiler flagsError reconfiguring sim-configmake: ***
[sim-config]
Error 2*
This still sounds like a compiler incompatibility. Somehow Cactus does not detect support for range based for statements, which however Intel 2013 claims to support in (search for "range-based", it is item N2930)
https://www.intel.com/content/www/us/en/developer/articles/technical/c0x-fea...
What does the file config.log (that autoconf points you to for detailed error messages) contain? It usually is something like configs/sim/config-data/config.log . The options used for CXXFLAGS (if those are the ones used) look fine to me.
*PART B:*
*Error: Product support for your (Comp-CL) license has expired.License file(s) used were (in this order): 1. Trusted Storage** 2.
Yes, this is a license issue. Note that you may still require -gxx-name options even for newer Intel compilers (they may default to the system g++ and system STL otherwise).
*Part C:* I compiled another ET successfully using the modules gcc-7.3.0, openmpi-3.1.4, FFTW3/3.3.3, gsl/1.16, openssl/1.1.1a, zlib/1.2.8, cmake/3.15.4, libjpeg/1.2.1, HDF5/1.8.10, openmpi/3.1.4.
But there seems to be a repetitive warning while buliding ET, which I am not sure if I should be worried about: */usr/bin/ld: warning: libgfortran.so.3, needed by /usr/lib64/atlas/liblapack.so, may conflict with libgfortran.so.4*
Basically: /usr/lib64/atlas/liblapack.so (the system ATLAS library) has been compiled with at version of gfortran much older than the one you are using. This can be fail in particular when involving strings being passed to Fortran code.
Please let me know if I should consider changing something to get rid of this warning, I have attached ini file (kanad_et8.ini), cfg file (kanad_et8.cfg) and full terminal output (out_et8.txt) for reference.
You may need to set:
LAPACK_DIR = BUILD BLAS_DIR = BUILD
to force the EinsteinToolkit to build its own (slow, but we do not rely on BLAS / LAPACK for speed) versions of LAPACK and BLAS (or you can try using OpenBLAS which is faster, but as said, speed of those two is not really relevant for typical ET simulations).
Now, having got this compiled successfully, should I continue to pursue compiling ET with intel compilers? Though I am still not sure if this ET (with gcc-7.3) will show up any errors in future as I aim to work on
binary
neutron star merger simulations. Please let me know your thoughts on
this.
Historically we did see slightly faster code with the Intel compiler. I suspect that similar speeds can be reached using GNU compilers by now though if one sets -ffast-math and similar options (that Intel defaults to) in CFLAGS and CXXFLAGS (Fortran has some of those optimizations allowed by the language already so it does not do quite so much for Fortran code).
See eg: https://gcc.gnu.org/wiki/FloatingPointMath for an explanation of the compromises this involves.
Yours, Roland
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .