[Users] Issue with compiling ET on cluster

Tue Nov 15 11:00:34 CST 2022

Hello Roland,

Thanks for pointing out the issue. I'll check with HPC admins to fix the
Intel-17 or install a better version. Could you please help me with the
issue regarding running BNSM on more than 1 node? The issue is explained in
my previous email in reply to your suggestions on problems divided into
Part A,  B and C.

Regards
Shamim

>
> <https://streaklinks.com/BR5Cd08uLuXq6QxdRQF3TX0k/https%3A%2F%2Fwww.cism.ucl.ac.be%2FServices%2FFormations%2FICS%2Fics_2013.0.028%2Fcomposer_xe_2013.1.117%2FDocumentation%2Fen_US%2FRelease_Notes_C_2013_L_EN.pdf>>
> > For Part C, I found a LAPACK library in our HPC, and the compilation
> > > process was completed without that warning. The Helloworld is also
> executed
> > > correctly.
> > >
> > > Currently, with this ET, I am facing issue on executing the gallery
> BNSM
> > > simulation on 2 nodes. The info in the command line goes as follows:
> > >
> > > *./simfactory/bin/sim create-submit nsns_p32_t24_8 --procs=32 --ppn=16
> > > --num-threads=1 --ppn-used=16 --num-smt=1 --parfile=par/nsnstohmns1.par
> > > --walltime=25:00:00*
> > >
> > > In the out file, the error is:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > *+ mpiexec -n 32 -npernode 16
> > > /home2/mallick/simulations2/nsns_p32_t24_11/SIMFACTORY/exe/cactus_sim
> -L 3
> > >
> /home2/mallick/simulations2/nsns_p32_t24_11/output-0000/nsnstohmns1.par--------------------------------------------------------------------------Your
> > > job has requested more processes than the ppr forthis topology can
> > > support:  App:
> > > /home2/mallick/simulations2/nsns_p32_t24_11/SIMFACTORY/exe/cactus_sim
> > > Number of procs:  32  PPR: 16:nodePlease revise the conflict and try
> > >
> again.--------------------------------------------------------------------------Simfactory
> > > Done at date: Mon 14 Nov 2022 07:11:57 PM IST*
> > >
> > > The simulations seem to work fine with 1 node, that is, --procs set to
> > > 16,8 or 4 keeping --ppn to 16. I assume the mpiexec command in
> runscript
> > > needs to be updated to provide clarity to the mpi library while using
> more
> > > than 1 node.
> > >
> > > I need some help here, I don't know how to proceed at this point. I am
> > > attaching the run, ini, cfg, sub script used for this ET and the output
> > > file for reference. Thanks in advance for all the help.
> > >
> > > Regards
> > > Shamim Haque
> > > Senior Research Fellow (SRF)
> > > Department of Physics
> > > IISER Bhopal
> > >
> > > ᐧ
> > >
> > > On Mon, Nov 14, 2022 at 8:22 PM Roland Haas <rhaas at illinois.edu>
> wrote:
> > >
> > >> Hello Shamim Haque,
> > >>
> > >> > *Part A:*
> > >> > Keeping compiler Intel 2013, I tried to add lines -gcc_name and
> > >> -gxx_name
> > >> > to CFLAGS and CXXFLAGS pointing to gcc/7.3.0.
> <https://streaklinks.com/BR5Cd08_vwoS01JRmAmyuuGa/http%3A%2F%2F7.3.0.>
> The error while
> > >> compiling is
> > >> > the following:
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > *checking for C++ lambda expressions... yeschecking for C++
> range-based
> > >> for
> > >> > statements... noCactus requires a C++11 compiler -- check your C++
> > >> compiler
> > >> > and C++ compiler flagsError reconfiguring sim-configmake: ***
> > >> [sim-config]
> > >> > Error 2*
> > >>
> > >> This still sounds like a compiler incompatibility. Somehow Cactus does
> > >> not detect support for range based for statements, which however Intel
> > >> 2013 claims to support in (search for "range-based", it is  item
> N2930)
> > >>
> > >>
> > >>
> https://urldefense.com/v3/__https://www.intel.com/content/www/us/en/developer/articles/technical/c0x-features-supported-by-intel-c-compiler.html__;!!DZ3fjg!4p6GfR3s9DU1DFu1e47xeUg3PBbxUi312fkmUSjyr8brzIS1vwSemU9s5aD35GvJj6eOhHeniZM3HGkosXBN$
> > >>
> > >> What does the file config.log (that autoconf points you to for
> detailed
> > >> error messages) contain? It usually is something like
> > >> configs/sim/config-data/config.log . The options used for CXXFLAGS (if
> > >> those are the ones used) look fine to me.
> > >>
> > >> > *PART B:*
> > >>
> > >> > *Error: Product support for your (Comp-CL) license has
> expired.License
> > >> > file(s) used were (in this order):    1.  Trusted Storage**  2.
> > >>
> > >> Yes, this is a license issue. Note that you may still require
> -gxx-name
> > >> options even for newer Intel compilers (they may default to the system
> > >> g++ and system STL otherwise).
> > >>
> > >> > *Part C:*
> > >> > I compiled another ET successfully using the modules gcc-7.3.0,
> > >> > openmpi-3.1.4, FFTW3/3.3.3, gsl/1.16, openssl/1.1.1a, zlib/1.2.8,
> > >> > cmake/3.15.4, libjpeg/1.2.1, HDF5/1.8.10,  openmpi/3.1.4.
> <https://streaklinks.com/BR5Cd08MjEDU_beQoAFbaxq9/http%3A%2F%2F3.1.4.>
> > >> >
> > >> > But there seems to be a repetitive warning while buliding ET, which
> I am
> > >> > not sure if I should be worried about:
> > >> > */usr/bin/ld: warning: libgfortran.so.3, needed by
> > >> > /usr/lib64/atlas/liblapack.so, may conflict with libgfortran.so.4*
> > >>
> > >> Basically: /usr/lib64/atlas/liblapack.so (the system ATLAS library)
> has
> > >> been compiled with at version of gfortran much older than the one you
> > >> are using. This can be fail in particular when involving strings being
> > >> passed to Fortran code.
> > >>
> > >> > Please let me know if I should consider changing something to get
> rid of
> > >> > this warning, I have attached ini file (kanad_et8.ini), cfg file
> > >> > (kanad_et8.cfg) and full terminal output (out_et8.txt) for
> reference.
> > >>
> > >> You may need to set:
> > >>
> > >> LAPACK_DIR = BUILD
> > >> BLAS_DIR = BUILD
> > >>
> > >> to force the EinsteinToolkit to build its own (slow, but we do not
> rely
> > >> on BLAS / LAPACK for speed) versions of LAPACK and BLAS (or you can
> try
> > >> using OpenBLAS which is faster, but as said, speed of those two is not
> > >> really relevant for typical ET simulations).
> > >>
> > >> > Now, having got this compiled successfully, should I continue to
> pursue
> > >> > compiling ET with intel compilers? Though I am still not sure if
> this ET
> > >> > (with gcc-7.3) will show up any errors in future as I aim to work on
> > >> binary
> > >> > neutron star merger simulations. Please let me know your thoughts on
> > >> this.
> > >>
> > >> Historically we did see slightly faster code with the Intel compiler.
> I
> > >> suspect that similar speeds can be reached using GNU compilers by now
> > >> though if one sets -ffast-math and similar options (that Intel
> > >> defaults to) in CFLAGS and CXXFLAGS (Fortran has some of those
> > >> optimizations allowed by the language already so it does not do quite
> > >> so much for Fortran code).
> > >>
> > >> See eg:
> https://urldefense.com/v3/__https://gcc.gnu.org/wiki/FloatingPointMath__;!!DZ3fjg!4p6GfR3s9DU1DFu1e47xeUg3PBbxUi312fkmUSjyr8brzIS1vwSemU9s5aD35GvJj6eOhHeniZM3HJ06ogmy$
> for an explanation
> > >> of the compromises this involves.
> > >>
> > >> Yours,
> > >> Roland
> > >>
> > >> --
> > >> My email is as private as my paper mail. I therefore support
> encrypting
> > >> and signing email messages. Get my PGP key from
> https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!4p6GfR3s9DU1DFu1e47xeUg3PBbxUi312fkmUSjyr8brzIS1vwSemU9s5aD35GvJj6eOhHeniZM3HE00Tvms$
> .
> > >>
> > >
>
> --
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://pgp.mit.edu .
>
ᐧ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20221115/a4a61021/attachment-0001.html 