Hello Roland,
Thanks for pointing out the issue. I'll check with HPC admins to fix the Intel-17 or install a better version. Could you please help me with the issue regarding running BNSM on more than 1 node? The issue is explained in my previous email in reply to your suggestions on problems divided into Part A, B and C.
Regards Shamim
For Part C, I found a LAPACK library in our HPC, and the compilation
process was completed without that warning. The Helloworld is also
executed
correctly.
Currently, with this ET, I am facing issue on executing the gallery
BNSM
simulation on 2 nodes. The info in the command line goes as follows:
*./simfactory/bin/sim create-submit nsns_p32_t24_8 --procs=32 --ppn=16 --num-threads=1 --ppn-used=16 --num-smt=1 --parfile=par/nsnstohmns1.par --walltime=25:00:00*
In the out file, the error is:
*+ mpiexec -n 32 -npernode 16 /home2/mallick/simulations2/nsns_p32_t24_11/SIMFACTORY/exe/cactus_sim
-L 3
/home2/mallick/simulations2/nsns_p32_t24_11/output-0000/nsnstohmns1.par--------------------------------------------------------------------------Your
job has requested more processes than the ppr forthis topology can support: App: /home2/mallick/simulations2/nsns_p32_t24_11/SIMFACTORY/exe/cactus_sim Number of procs: 32 PPR: 16:nodePlease revise the conflict and try
again.--------------------------------------------------------------------------Simfactory
Done at date: Mon 14 Nov 2022 07:11:57 PM IST*
The simulations seem to work fine with 1 node, that is, --procs set to 16,8 or 4 keeping --ppn to 16. I assume the mpiexec command in
runscript
needs to be updated to provide clarity to the mpi library while using
more
than 1 node.
I need some help here, I don't know how to proceed at this point. I am attaching the run, ini, cfg, sub script used for this ET and the output file for reference. Thanks in advance for all the help.
Regards Shamim Haque Senior Research Fellow (SRF) Department of Physics IISER Bhopal
ᐧ
On Mon, Nov 14, 2022 at 8:22 PM Roland Haas rhaas@illinois.edu
wrote:
Hello Shamim Haque,
*Part A:* Keeping compiler Intel 2013, I tried to add lines -gcc_name and
-gxx_name
to CFLAGS and CXXFLAGS pointing to gcc/7.3.0.
https://streaklinks.com/BR5Cd08_vwoS01JRmAmyuuGa/http%3A%2F%2F7.3.0. The error while
compiling is
the following:
*checking for C++ lambda expressions... yeschecking for C++
range-based
for
statements... noCactus requires a C++11 compiler -- check your C++
compiler
and C++ compiler flagsError reconfiguring sim-configmake: ***
[sim-config]
Error 2*
This still sounds like a compiler incompatibility. Somehow Cactus does not detect support for range based for statements, which however Intel 2013 claims to support in (search for "range-based", it is item
N2930)
https://urldefense.com/v3/__https://www.intel.com/content/www/us/en/develope...
What does the file config.log (that autoconf points you to for
detailed
error messages) contain? It usually is something like configs/sim/config-data/config.log . The options used for CXXFLAGS (if those are the ones used) look fine to me.
*PART B:*
*Error: Product support for your (Comp-CL) license has
expired.License
file(s) used were (in this order): 1. Trusted Storage** 2.
Yes, this is a license issue. Note that you may still require
-gxx-name
options even for newer Intel compilers (they may default to the system g++ and system STL otherwise).
*Part C:* I compiled another ET successfully using the modules gcc-7.3.0, openmpi-3.1.4, FFTW3/3.3.3, gsl/1.16, openssl/1.1.1a, zlib/1.2.8, cmake/3.15.4, libjpeg/1.2.1, HDF5/1.8.10, openmpi/3.1.4.
https://streaklinks.com/BR5Cd08MjEDU_beQoAFbaxq9/http%3A%2F%2F3.1.4.
But there seems to be a repetitive warning while buliding ET, which
I am
not sure if I should be worried about: */usr/bin/ld: warning: libgfortran.so.3, needed by /usr/lib64/atlas/liblapack.so, may conflict with libgfortran.so.4*
Basically: /usr/lib64/atlas/liblapack.so (the system ATLAS library)
has
been compiled with at version of gfortran much older than the one you are using. This can be fail in particular when involving strings being passed to Fortran code.
Please let me know if I should consider changing something to get
rid of
this warning, I have attached ini file (kanad_et8.ini), cfg file (kanad_et8.cfg) and full terminal output (out_et8.txt) for
reference.
You may need to set:
LAPACK_DIR = BUILD BLAS_DIR = BUILD
to force the EinsteinToolkit to build its own (slow, but we do not
rely
on BLAS / LAPACK for speed) versions of LAPACK and BLAS (or you can
try
using OpenBLAS which is faster, but as said, speed of those two is not really relevant for typical ET simulations).
Now, having got this compiled successfully, should I continue to
pursue
compiling ET with intel compilers? Though I am still not sure if
this ET
(with gcc-7.3) will show up any errors in future as I aim to work on
binary
neutron star merger simulations. Please let me know your thoughts on
this.
Historically we did see slightly faster code with the Intel compiler.
I
suspect that similar speeds can be reached using GNU compilers by now though if one sets -ffast-math and similar options (that Intel defaults to) in CFLAGS and CXXFLAGS (Fortran has some of those optimizations allowed by the language already so it does not do quite so much for Fortran code).
See eg:
https://urldefense.com/v3/__https://gcc.gnu.org/wiki/FloatingPointMath__;!!D... for an explanation
of the compromises this involves.
Yours, Roland
-- My email is as private as my paper mail. I therefore support
encrypting
and signing email messages. Get my PGP key from
https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!4p6GfR3s9DU1DFu1e4... .
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .
ᐧ