<div dir="ltr"><div class="gmail_default" style="color:#000000">Hello Roland,</div><div class="gmail_default" style="color:#000000"><br></div><div class="gmail_default" style="color:#000000">Thanks for the comments on the issues. I have attached the config.log file for Part A, and I'll see what can be done about Part B. </div><div class="gmail_default" style="color:#000000"><br></div><div class="gmail_default" style="color:#000000">For Part C, I found a LAPACK library in our HPC, and the compilation process was completed without that warning. The Helloworld is also executed correctly. </div><div class="gmail_default" style="color:#000000"><br></div><div class="gmail_default" style="color:#000000">Currently, with this ET, I am facing issue on executing the gallery BNSM simulation on 2 nodes. The info in the command line goes as follows:</div><div class="gmail_default" style="color:#000000"><br></div><div class="gmail_default"><b><i><font color="#a64d79">./simfactory/bin/sim create-submit nsns_p32_t24_8 --procs=32 --ppn=16 --num-threads=1 --ppn-used=16 --num-smt=1 --parfile=par/nsnstohmns1.par --walltime=25:00:00</font></i></b><br></div><div><div dir="ltr" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><font color="#666666"><br></font></div><div><div class="gmail_default" style="color:rgb(0,0,0)">In the out file, the error is:</div><div class="gmail_default" style="color:rgb(0,0,0)"><br></div><b><i><font color="#a64d79"><span class="gmail_default"></span>+ mpiexec -n 32 -npernode 16 /home2/mallick/simulations2/nsns_p32_t24_11/SIMFACTORY/exe/cactus_sim -L 3 /home2/mallick/simulations2/nsns_p32_t24_11/output-0000/nsnstohmns1.par<br>--------------------------------------------------------------------------<br>Your job has requested more processes than the ppr for<br>this topology can support:<br> App: /home2/mallick/simulations2/nsns_p32_t24_11/SIMFACTORY/exe/cactus_sim<br> Number of procs: 32<br> PPR: 16:node<br>Please revise the conflict and try again.<br>--------------------------------------------------------------------------<br>Simfactory Done at date: Mon 14 Nov 2022 07:11:57 PM IST</font></i></b><span style="color:rgb(0,0,0)"></span><b><i><font color="#a64d79"><br></font></i></b><br></div><div><font color="#000000">T<span class="gmail_default" style="color:rgb(0,0,0)">he simulations seem to work fine with 1 node, that is, --procs set to 16,8 or 4 keeping --ppn to 16. I assume the mpiexec command in runscript needs to be updated to provide clarity to the mpi library while using more than 1 node.</span></font></div><div><font color="#000000"><span class="gmail_default" style="color:rgb(0,0,0)"><br></span></font></div><div><font color="#000000"><span class="gmail_default" style="color:rgb(0,0,0)">I need some help here, I don't know how to proceed at this point. I am attaching the run, ini, cfg, sub script used for this ET and the output file for reference. </span></font><span style="color:rgb(0,0,0)">Thanks in advance for all the help.</span></div><div><font color="#000000"><span class="gmail_default" style="color:rgb(0,0,0)"><br></span></font></div><div><font color="#000000"><span class="gmail_default" style="color:rgb(0,0,0)">Regards</span></font></div><div dir="ltr"><font color="#666666">Shamim Haque</font></div><div dir="ltr"><font color="#666666">Senior Research Fellow (SRF)<br></font><div><font color="#666666">Department of Physics</font></div><div><font color="#666666">IISER Bhopal</font></div></div></div></div></div></div></div><br></div><div hspace="streak-pt-mark" style="max-height:1px"><img alt="" style="width:0px;max-height:0px;overflow:hidden" src="https://mailfoogae.appspot.com/t?sender=ac2hhbWltc0BpaXNlcmIuYWMuaW4%3D&type=zerocontent&guid=62bd8ea4-f8fb-4312-9cdf-2b63ad116dfa"><font color="#ffffff" size="1">ᐧ</font></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Nov 14, 2022 at 8:22 PM Roland Haas <<a href="mailto:rhaas@illinois.edu" target="_blank">rhaas@illinois.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello Shamim Haque,<br>
<br>
> *Part A:*<br>
> Keeping compiler Intel 2013, I tried to add lines -gcc_name and -gxx_name<br>
> to CFLAGS and CXXFLAGS pointing to gcc/<a href="http://7.3.0." rel="noreferrer" target="_blank">7.3.0.</a> The error while compiling is<br>
> the following:<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> *checking for C++ lambda expressions... yeschecking for C++ range-based for<br>
> statements... noCactus requires a C++11 compiler -- check your C++ compiler<br>
> and C++ compiler flagsError reconfiguring sim-configmake: *** [sim-config]<br>
> Error 2*<br>
<br>
This still sounds like a compiler incompatibility. Somehow Cactus does<br>
not detect support for range based for statements, which however Intel<br>
2013 claims to support in (search for "range-based", it is item N2930)<br>
<br>
<a href="https://www.intel.com/content/www/us/en/developer/articles/technical/c0x-features-supported-by-intel-c-compiler.html" rel="noreferrer" target="_blank">https://www.intel.com/content/www/us/en/developer/articles/technical/c0x-features-supported-by-intel-c-compiler.html</a><br>
<br>
What does the file config.log (that autoconf points you to for detailed<br>
error messages) contain? It usually is something like<br>
configs/sim/config-data/config.log . The options used for CXXFLAGS (if<br>
those are the ones used) look fine to me.<br>
<br>
> *PART B:*<br>
<br>
> *Error: Product support for your (Comp-CL) license has expired.License<br>
> file(s) used were (in this order): 1. Trusted Storage** 2.<br>
<br>
Yes, this is a license issue. Note that you may still require -gxx-name<br>
options even for newer Intel compilers (they may default to the system<br>
g++ and system STL otherwise).<br>
<br>
> *Part C:*<br>
> I compiled another ET successfully using the modules gcc-7.3.0,<br>
> openmpi-3.1.4, FFTW3/3.3.3, gsl/1.16, openssl/1.1.1a, zlib/1.2.8,<br>
> cmake/3.15.4, libjpeg/1.2.1, HDF5/1.8.10, openmpi/<a href="http://3.1.4." rel="noreferrer" target="_blank">3.1.4.</a><br>
> <br>
> But there seems to be a repetitive warning while buliding ET, which I am<br>
> not sure if I should be worried about:<br>
> */usr/bin/ld: warning: libgfortran.so.3, needed by<br>
> /usr/lib64/atlas/liblapack.so, may conflict with libgfortran.so.4*<br>
<br>
Basically: /usr/lib64/atlas/liblapack.so (the system ATLAS library) has<br>
been compiled with at version of gfortran much older than the one you<br>
are using. This can be fail in particular when involving strings being<br>
passed to Fortran code.<br>
<br>
> Please let me know if I should consider changing something to get rid of<br>
> this warning, I have attached ini file (kanad_et8.ini), cfg file<br>
> (kanad_et8.cfg) and full terminal output (out_et8.txt) for reference.<br>
<br>
You may need to set:<br>
<br>
LAPACK_DIR = BUILD<br>
BLAS_DIR = BUILD<br>
<br>
to force the EinsteinToolkit to build its own (slow, but we do not rely<br>
on BLAS / LAPACK for speed) versions of LAPACK and BLAS (or you can try<br>
using OpenBLAS which is faster, but as said, speed of those two is not<br>
really relevant for typical ET simulations).<br>
<br>
> Now, having got this compiled successfully, should I continue to pursue<br>
> compiling ET with intel compilers? Though I am still not sure if this ET<br>
> (with gcc-7.3) will show up any errors in future as I aim to work on binary<br>
> neutron star merger simulations. Please let me know your thoughts on this.<br>
<br>
Historically we did see slightly faster code with the Intel compiler. I<br>
suspect that similar speeds can be reached using GNU compilers by now<br>
though if one sets -ffast-math and similar options (that Intel<br>
defaults to) in CFLAGS and CXXFLAGS (Fortran has some of those<br>
optimizations allowed by the language already so it does not do quite<br>
so much for Fortran code).<br>
<br>
See eg: <a href="https://gcc.gnu.org/wiki/FloatingPointMath" rel="noreferrer" target="_blank">https://gcc.gnu.org/wiki/FloatingPointMath</a> for an explanation<br>
of the compromises this involves.<br>
<br>
Yours,<br>
Roland<br>
<br>
-- <br>
My email is as private as my paper mail. I therefore support encrypting<br>
and signing email messages. Get my PGP key from <a href="http://pgp.mit.edu" rel="noreferrer" target="_blank">http://pgp.mit.edu</a> .<br>
</blockquote></div>