[Users] Compilation Error in cluster ParamGanga

Roland Haas rhaas at illinois.edu
Tue Jun 6 08:25:43 CDT 2023


Hello Shamim Haque,

"icpc unable to checkout FLEXlm license" is caused by the compiler
being unable to contact the license server. This is unrelated to the
C++ compilation issue really.

To be sure:

* the only thing you did was add
  "-gxx-name=/opt/ohpc/pub/compiler/gcc/8.3.0/bin/g++" was it? In
  particular you did not add / remove any "module load" commands (e.g.,
  added a module load gcc after an existing module load intel)? I am
  asking b/c module load gcc may unload the intel compiler module and
  unset the environment variable that the Intel compiler uses to
  determine how to contact the license server.

* you may also try to compile with just "make -j1" or
  "simfactory/bin/sim build -j1" to use only a single compiler instance
  in case the number of compiler licenses available is limited or the
  license server overwhelmed by too many concurrent compilations (by
  you or other users of the cluster)

The issue with fftw3 and using gcc is that fftw3 (you are using a
system provided fftw3? Either from a module or otherwise provided by
the admin) has been compiled with the Intel compiler and thus cannot be
used with the GNU compiler. You an set "FFTW_DIR = BUILD" to make the
Einstein Toolkit build a copy from scratch using an included tar file.
Not that this may be the case for other libraries as well. In
particular you usually need to ensure that the MPI library used was
compiled with the same compiler you use to compile the simulation code.

Yours,
Roland

> Hello Roland,
> 
> Thank you for the reply.
> 
> I pointed -gxx-name to gcc/8.3.0, which resulted in an error related to the
> "icpc unable to checkout FLEXlm license". I also tried to use a
> different intel compiler (and point -gxx-name to gcc/11.1.0) to check if
> the error can go away, but it remains the same. I communicated this to the
> HPC admin and awaiting their response; please let me know if I should fix
> something in my configuration. I have attached the out file
> (intel_compile.out), config log (intel_config.log),
> and optionlist (intel.cfg) for reference.
> 
> Meanwhile, I also attempted to configure ET with gcc/11.1.0, which resulted
> in some issues regarding the FFTW3. I have contacted the admins regarding
> this, but please let me know if it is something to be fixed within the
> optionlist. I have attached the out file (gcc_compile.out), config log
> (gcc_config.log), and optionlist (gcc.cfg) for reference.
> 
> The error in the output file says the following:
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> *Formaline:  is still running, e.g. the final link stage.)/usr/bin/ld:
> warning: libimf.so, needed by /home/apps/fftw3/fftw3.3.8/lib/libfftw3.so,
> not found (try using -rpath or -rpath-link)/usr/bin/ld: warning:
> libsvml.so, needed by /home/apps/fftw3/fftw3.3.8/lib/libfftw3.so, not found
> (try using -rpath or -rpath-link)/usr/bin/ld: warning: libirng.so, needed
> by /home/apps/fftw3/fftw3.3.8/lib/libfftw3.so, not found (try using -rpath
> or -rpath-link)/usr/bin/ld: warning: libintlc.so.5, needed by
> /home/apps/fftw3/fftw3.3.8/lib/libfftw3.so, not found (try using -rpath or
> -rpath-link)/usr/bin/ld: warning: libgfortran.so.3, needed by
> /lib/../lib64/liblapack.so, may conflict with
> libgfortran.so.5/home/apps/fftw3/fftw3.3.8/lib/libfftw3.so: undefined
> reference to
> `__intel_cpu_features_init_x'/home/apps/fftw3/fftw3.3.8/lib/libfftw3.so:
> undefined reference to
> `__svml_sincos2'/home/apps/fftw3/fftw3.3.8/lib/libfftw3.so: undefined
> reference to
> `_intel_fast_memset'/home/apps/fftw3/fftw3.3.8/lib/libfftw3.so: undefined
> reference to
> `__intel_cpu_feature_indicator_x'/home/apps/fftw3/fftw3.3.8/lib/libfftw3.so:
> undefined reference to
> `_intel_fast_memmove'/home/apps/fftw3/fftw3.3.8/lib/libfftw3.so: undefined
> reference to `_intel_fast_memcpy'*
> 
> Thanks in advance for all the help.
> 
> Regards
> Shamim
> 
>> 
> On Fri, Jun 2, 2023 at 1:31 AM Roland Haas <rhaas at illinois.edu> wrote:
> 
> > Hello Shamim Haque,
> >
> > This most likely is caused by not having a full C++11 software stack
> > enabled.
> >
> > The Intel compiler always uses the C++ STL library of the GNU g++
> > compiler and must be matched up with a compatible g++ compiler's STL
> > to correctly support C++11.
> >
> > This ticket (the last of many, hopefully) contains details:
> >
> >
> > https://urldefense.com/v3/__https://bitbucket.org/einsteintoolkit/tickets/issues/2663/add-an-notice-about-minimum-intel-gcc__;!!DZ3fjg!4Z697XBz_Q8JIQSrOykhIqb_thU0tH0XLyTUT1zDMwhZlk1earaaKDJXweGCrJ3LGb2HvWAHbeRBlRGRwlyl$ 
> >
> > Basically, the fix is to find a gcc compiler at least version 6 on your
> > cluster, then pass the option
> >
> > -gxx-name PATH-TO-GXX
> >
> > to icpc in the CXXFLAGS option list variable (see eg
> >
> > https://urldefense.com/v3/__https://bitbucket.org/simfactory/simfactory2/src/7e2592d264b1afe9ca2cb54bcab8dd9e4165cf98/mdb/optionlists/smic-openmpi.cfg?at=master*smic-openmpi.cfg-15:16__;Iw!!DZ3fjg!4Z697XBz_Q8JIQSrOykhIqb_thU0tH0XLyTUT1zDMwhZlk1earaaKDJXweGCrJ3LGb2HvWAHbeRBlSBbGTFf$ 
> > ).
> >
> > If there is no gcc compiler version 6 or newer installed in the system,
> > you will have to contact the administrators and ask them about it.
> >
> > The demo code in the description of
> > https://urldefense.com/v3/__https://bitbucket.org/cactuscode/cactus/pull-requests/152__;!!DZ3fjg!4Z697XBz_Q8JIQSrOykhIqb_thU0tH0XLyTUT1zDMwhZlk1earaaKDJXweGCrJ3LGb2HvWAHbeRBlUmecdAQ$  may help
> > demonstrate to the admins what the issue is.
> >
> > Yours,
> > Roland
> >  
> > > Hello All,
> > >
> > > I have made little progress on compiling ETK on this cluster, as the  
> > admins  
> > > allowed me to compile ETK on the login node. I am finding new errors and  
> > am  
> > > unable to debug the issue. I am trying to configure ETK using Intel19,  
> > and  
> > > the error says the following:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >  
> > */home/shamims.iiserb/ET_debug/Cactus/arrangements/Carpet/CarpetMask/src/mask_surface.cc(160):  
> > > error: more than one instance of overloaded function "CarpetMask::isnan"
> > > matches the argument list:            function "isnan(double)"
> > > function "std::isnan(double)"            argument types are: (CCTK_REAL8)
> > >             assert(not isnan(phi));              ^compilation aborted for
> > >  
> > /home/shamims.iiserb/ET_debug/Cactus/configs/sim/build/CarpetMask/mask_surface.cc  
> > > (code 2)make[3]: *** [mask_surface.cc.o] Error 2make[2]: ***  
> > [make.checked]  
> > > Error 2make[1]: ***
> > >  
> > [/home/shamims.iiserb/ET_debug/Cactus/configs/sim/lib/libthorn_CarpetMask.a]  
> > > Error 2make: *** [sim] Error 2*
> > >
> > > I have attached the machinefile, optionlist, config log and full output
> > > file for reference. I configured the scripts using the available example
> > > optionlist and machinefile for supermuc_ng and stampede2. Please let me
> > > know if more info is required, and thanks in advance for your help.
> > >
> > > Regards
> > > Shamim Haque
> > > Senior Research Fellow (SRF)
> > > Department of Physics
> > > IISER Bhopal
> > >
> > > ᐧ
> > >
> > > On Fri, May 19, 2023 at 2:32 PM Shamim Haque 1910511 <
> > shamims at iiserb.ac.in>
> > > wrote:
> > >  
> > > > Hello Roland,
> > > >
> > > > Thank you for the reply. Yes, I am trying to compile ETK in the  
> > computing  
> > > > node via sbatch queue. I am in touch with the system admin to resolve
> > > > the issue with the compilers.
> > > >
> > > > Regards
> > > > Shamim Haque
> > > > Senior Research Fellow (SRF)
> > > > Department of Physics
> > > > IISER Bhopal
> > > >
> > > > ᐧ
> > > >
> > > > On Thu, May 18, 2023 at 7:32 PM Roland Haas <rhaas at illinois.edu>  
> > wrote:  
> > > >  
> > > >> Hello Shamin,
> > > >>
> > > >> A missing math.h would point to a faulty compiler installation. Since
> > > >> you are trying to compile (are you?) ona compute node in a running  
> > job,  
> > > >> my guess would be that the cluster admins are not providing compilers
> > > >> on the compute nodes but only on the login nodes.
> > > >>
> > > >> This would be something to contact your cluster's help desk about.
> > > >>
> > > >> Yours,
> > > >> Roland
> > > >>  
> > > >> > Hello Roland,
> > > >> >
> > > >> > I did try that fix; the error remains the same. Actually, we landed  
> > on  
> > > >> an  
> > > >> > even more primitive problem, where Steve suggested a basic test,  
> > that  
> > > >> is to  
> > > >> > compile the following code:
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> > *#include <math.h>#include <stdio.h>int main() {    printf("%f\n",
> > > >> > M_PI);    return 0;}*
> > > >> >
> > > >> > This program compiles just fine in the login node but gives out the
> > > >> > following error while compiling it via jobscript submission in the  
> > > >> sbatch  
> > > >> > queue:
> > > >> >
> > > >> > prog.cpp:1:18: fatal error: math.h: No such file or directory
> > > >> >  #include <math.h>
> > > >> >                   ^
> > > >> > compilation terminated.
> > > >> >
> > > >> > ----OR-----
> > > >> >
> > > >> >  
> > > >>  
> > /home/apps/gcc-10.2.0/include/c++/10.2.0/x86_64-pc-linux-gnu/bits/os_defines.h:39:10:
> >  
> > > >> > fatal error: features.h: No such file or directory
> > > >> >    39 | #include <features.h>
> > > >> >       |          ^~~~~~~~~~~~
> > > >> > compilation terminated.
> > > >> >
> > > >> > I have reported this problem to the HPC admin and await their  
> > response.  
> > > >> >
> > > >> > Regards
> > > >> > Shamim Haque
> > > >> > Senior Research Fellow (SRF)
> > > >> > Department of Physics
> > > >> > IISER Bhopal
> > > >> >
> > > >> > ᐧ
> > > >> >
> > > >> > On Fri, May 12, 2023 at 12:20 AM Roland Haas <rhaas at illinois.edu>  
> > > >> wrote:  
> > > >> >  
> > > >> > > Hello all,
> > > >> > >
> > > >> > > Since the error is a missing M_PI and the configure script already
> > > >> > > gives a suggested fix:
> > > >> > >
> > > >> > > Try adding -D_XOPEN_SOURCE to CPPFLAGS.
> > > >> > >
> > > >> > > have you tried adding:
> > > >> > >
> > > >> > > CPPFLAGS=-D_XOPEN_SOURCE
> > > >> > >
> > > >> > > to your option list file before compiling?
> > > >> > >
> > > >> > > Yours,
> > > >> > > Roland
> > > >> > >  
> > > >> > > > Thanks Steve. I have raised complaint to the admin about this  
> > issue.  
> > > >> > > >
> > > >> > > > On Thu, May 11, 2023, 11:18 PM Steven R. Brandt <  
> > > >> sbrandt at cct.lsu.edu>  
> > > >> > > wrote:  
> > > >> > > >  
> > > >> > > > > I would talk to the system administrators about that. There  
> > is  
> > > >> > > definitely  
> > > >> > > > > something wrong with your compiler install if you can't find  
> > > >> math.h.  
> > > >> > > > >
> > > >> > > > > --Steve
> > > >> > > > > On 5/11/2023 2:02 AM, Shamim Haque 1910511 wrote:
> > > >> > > > >
> > > >> > > > > Dear Steve,
> > > >> > > > >
> > > >> > > > > I just rechecked the cpp program to output M_PI in the  
> > computing  
> > > >> nodes,  
> > > >> > > > > and that is not working right now, which was somehow working  
> > fine  
> > > >> a few  
> > > >> > > > > days ago. I also tried different gcc versions (4.8, 10.2,  
> > 11.1,  
> > > >> > > gnu8/8.3)  
> > > >> > > > > available in the cluster. All report similar issue:
> > > >> > > > >
> > > >> > > > > prog.cpp:1:18: fatal error: math.h: No such file or directory
> > > >> > > > >  #include <math.h>
> > > >> > > > >                   ^
> > > >> > > > > compilation terminated.
> > > >> > > > >
> > > >> > > > > ----OR-----
> > > >> > > > >
> > > >> > > > >  
> > > >> > >  
> > > >>  
> > /home/apps/gcc-10.2.0/include/c++/10.2.0/x86_64-pc-linux-gnu/bits/os_defines.h:39:10:  
> > > >>  
> > > >> > > > > fatal error: features.h: No such file or directory
> > > >> > > > >    39 | #include <features.h>
> > > >> > > > >       |          ^~~~~~~~~~~~
> > > >> > > > > compilation terminated.
> > > >> > > > >
> > > >> > > > > The cpp code seems to work fine in login node, as it gives  
> > out  
> > > >> the  
> > > >> > > value  
> > > >> > > > > of pi upon execution. I am contacting the hpc admin to  
> > resolve  
> > > >> this  
> > > >> > > issue.  
> > > >> > > > >
> > > >> > > > > I am attaching the test code (prog.cpp), the sbatch  
> > jobscript  
> > > >> (prog.sh)  
> > > >> > > > > and the output file (slurn-117473.out), if you would like to  
> > have  
> > > >> a  
> > > >> > > look at  
> > > >> > > > > it.
> > > >> > > > >
> > > >> > > > > Please let me know if I should try something different to  
> > get  
> > > >> this  
> > > >> > > working.  
> > > >> > > > >
> > > >> > > > > Regards
> > > >> > > > > Shamim Haque
> > > >> > > > > Senior Research Fellow (SRF)
> > > >> > > > > Department of Physics
> > > >> > > > > IISER Bhopal
> > > >> > > > >
> > > >> > > > > ᐧ
> > > >> > > > >
> > > >> > > > > On Mon, May 8, 2023 at 7:25 PM Shamim Haque 1910511 <  
> > > >> > > shamims at iiserb.ac.in>  
> > > >> > > > > wrote:
> > > >> > > > >  
> > > >> > > > >> Thanks Steve. I'll make the changes and try again. I'll let  
> > you  
> > > >> know  
> > > >> > > the  
> > > >> > > > >> outcome.
> > > >> > > > >>
> > > >> > > > >> Regards
> > > >> > > > >> Shamim Haque
> > > >> > > > >> Senior Research Fellow (SRF)
> > > >> > > > >> Department of Physics
> > > >> > > > >> IISER Bhopal
> > > >> > > > >>
> > > >> > > > >> ᐧ
> > > >> > > > >>
> > > >> > > > >> On Mon, May 8, 2023 at 7:21 PM Steven R. Brandt <  
> > > >> sbrandt at cct.lsu.edu>  
> > > >> > > > >> wrote:
> > > >> > > > >>  
> > > >> > > > >>> That tends to make me think you aren't using the compiler  
> > you  
> > > >> think  
> > > >> > > > >>> you're using. You could put full paths in the .cfg files  
> > and  
> > > >> > > recompile from  
> > > >> > > > >>> scratch.
> > > >> > > > >>>
> > > >> > > > >>> i.e.
> > > >> > > > >>>
> > > >> > > > >>> CXX = /full/path/to/icc
> > > >> > > > >>>
> > > >> > > > >>> --Steve
> > > >> > > > >>> On 5/3/2023 3:26 PM, Shamim Haque 1910511 wrote:
> > > >> > > > >>>
> > > >> > > > >>> Yes, this gives the value of Pi successfully, before  
> > starting  
> > > >> the  
> > > >> > > > >>> compilation:
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>> */home/vkerni.iitr/sam/trial 3.141593 Current machine:  
> > > >> paramganga3  
> > > >> > > Using  
> > > >> > > > >>> configuration: sim Reconfiguring sim Writing configuration  
> > to:  
> > > >> > > > >>>  
> > /home/vkerni.iitr/sam/ET_debug/Cactus/configs/sim/OptionList  
> > > >> Cactus -  
> > > >> > > > >>> version: 4.12.0 Reconfiguring sim. *
> > > >> > > > >>> *.*
> > > >> > > > >>> *.*
> > > >> > > > >>> *.*
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>> *checking for M_PI... no configure: error: M_PI not  
> > defined.  
> > > >> Try  
> > > >> > > adding  
> > > >> > > > >>> -D_XOPEN_SOURCE to CPPFLAGS. Error reconfiguring  
> > sim-config  
> > > >> make: ***  
> > > >> > > > >>> [sim-config] Error 2 *
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>> Shamim Haque
> > > >> > > > >>> Senior Research Fellow (SRF)
> > > >> > > > >>> Department of Physics
> > > >> > > > >>> IISER Bhopal
> > > >> > > > >>>
> > > >> > > > >>> ᐧ
> > > >> > > > >>>
> > > >> > > > >>> On Thu, May 4, 2023 at 12:29 AM Steven Brandt <  
> > > >> sbrandt at cct.lsu.edu>  
> > > >> > > > >>> wrote:
> > > >> > > > >>>  
> > > >> > > > >>>> So, sanity check, can you compile the following program  
> > > >> outside of  
> > > >> > > > >>>> Cactus?
> > > >> > > > >>>>
> > > >> > > > >>>> #include <math.h>
> > > >> > > > >>>> #include <stdio.h>
> > > >> > > > >>>>
> > > >> > > > >>>> int main() {
> > > >> > > > >>>>     printf("%f\n", M_PI);
> > > >> > > > >>>>     return 0;
> > > >> > > > >>>> }
> > > >> > > > >>>> On 5/3/2023 1:50 PM, Shamim Haque 1910511 wrote:
> > > >> > > > >>>>
> > > >> > > > >>>> Dear Steve,
> > > >> > > > >>>>
> > > >> > > > >>>> I tried gcc (10 and 11) as well and I am stuck with the  
> > > >> following  
> > > >> > > error:  
> > > >> > > > >>>> *configure: error: M_PI not defined. Try adding  
> > > >> -D_XOPEN_SOURCE to  
> > > >> > > > >>>> CPPFLAGS.*
> > > >> > > > >>>>
> > > >> > > > >>>> This also happened for Intel (2020 and 2019). I have  
> > attached  
> > > >> both  
> > > >> > > the  
> > > >> > > > >>>> output files here.
> > > >> > > > >>>>
> > > >> > > > >>>> Please ignore the out file in the first email, I have  
> > > >> uploaded  
> > > >> > > another  
> > > >> > > > >>>> out file by mistake. The rest of the files are correct in  
> > the  
> > > >> first  
> > > >> > > mail.  
> > > >> > > > >>>>
> > > >> > > > >>>> Regards
> > > >> > > > >>>> Shamim Haque
> > > >> > > > >>>> Senior Research Fellow (SRF)
> > > >> > > > >>>> Department of Physics
> > > >> > > > >>>> IISER Bhopal
> > > >> > > > >>>>
> > > >> > > > >>>> ᐧ
> > > >> > > > >>>>
> > > >> > > > >>>> On Wed, May 3, 2023 at 8:04 PM Steven Brandt <  
> > > >> sbrandt at cct.lsu.edu>  
> > > >> > > > >>>> wrote:
> > > >> > > > >>>>  
> > > >> > > > >>>>> I notice from the output that your compiler claims  
> > "limits.h"  
> > > >> is  
> > > >> > > > >>>>> missing. Possibly there's something wrong with your icc  
> > > >> setup?  
> > > >> > > Maybe you  
> > > >> > > > >>>>> should try using gcc / gfortran instead.
> > > >> > > > >>>>>
> > > >> > > > >>>>> --Steve
> > > >> > > > >>>>> On 5/3/2023 8:20 AM, Shamim Haque 1910511 wrote:
> > > >> > > > >>>>>
> > > >> > > > >>>>> Dear Frank,
> > > >> > > > >>>>>
> > > >> > > > >>>>> I added the alias pattern for the computing nodes and now  
> > ETK  
> > > >> can  
> > > >> > > > >>>>> detect the cluster (checked by asking "whoami") during  
> > the  
> > > >> batch  
> > > >> > > queuing.  
> > > >> > > > >>>>> I'll get in touch with the admins to include 'patch'.
> > > >> > > > >>>>>
> > > >> > > > >>>>> However, the problem still persists, having assured that  
> > the  
> > > >> > > correct  
> > > >> > > > >>>>> machine file and option list are being used during the  
> > > >> compilation.  
> > > >> > > > >>>>>
> > > >> > > > >>>>> Regards
> > > >> > > > >>>>> Shamim Haque
> > > >> > > > >>>>> Senior Research Fellow (SRF)
> > > >> > > > >>>>> Department of Physics
> > > >> > > > >>>>> IISER Bhopal
> > > >> > > > >>>>>
> > > >> > > > >>>>> ᐧ
> > > >> > > > >>>>>
> > > >> > > > >>>>> On Wed, May 3, 2023 at 3:49 PM Frank Loeffler <
> > > >> > > > >>>>> frank.loeffler at uni-jena.de> wrote:
> > > >> > > > >>>>>  
> > > >> > > > >>>>>> Hi Shamim
> > > >> > > > >>>>>>
> > > >> > > > >>>>>> On Wed, May 03, 2023 at 03:11:49PM +0530, Shamim Haque  
> > > >> 1910511  
> > > >> > > wrote:  
> > > >> > > > >>>>>> >We are trying to compile ETK on ParamGanga at IIT  
> > Roorkee.  
> > > >> The  
> > > >> > > > >>>>>> >compilation
> > > >> > > > >>>>>> >stops at a very initial step and gives out the error:
> > > >> > > > >>>>>> >
> > > >> > > > >>>>>> >*checking whether the Fortran compiler (gfortran  )  
> > > >> works...  
> > > >> > > > >>>>>> yeschecking  
> > > >> > > > >>>>>> >whether the Fortran compiler (gfortran  ) is a  
> > > >> > > cross-compiler...  
> > > >> > > > >>>>>> nochecking  
> > > >> > > > >>>>>> >whether the Fortran compiler (gfortran  ) supports  
> > TYPE(*)  
> > > >> for  
> > > >> > > > >>>>>> >CCTK_PointerTo... yeschecking how to run the C  
> > > >> preprocessor...  
> > > >> > > > >>>>>> cppchecking  
> > > >> > > > >>>>>> >for ANSI C header files... nochecking for C99  
> > features...  
> > > >> > > > >>>>>> yeschecking for  
> > > >> > > > >>>>>> >M_PI... noconfigure: error: M_PI not defined. Try  
> > adding  
> > > >> > > > >>>>>> -D_XOPEN_SOURCE to  
> > > >> > > > >>>>>> >CPPFLAGS.*
> > > >> > > > >>>>>> >
> > > >> > > > >>>>>> >We tried adding -D_XOPEN_SOURCE to CPPFLAGS, but it  
> > does  
> > > >> not  
> > > >> > > help.  
> > > >> > > > >>>>>> This  
> > > >> > > > >>>>>> >error is consistent if we set up using Intel Compiler  
> > (2020  
> > > >> or  
> > > >> > > > >>>>>> 2019), or  
> > > >> > > > >>>>>> >gcc (11 or 10). I have attached the outfile, error  
> > file,  
> > > >> > > machine  
> > > >> > > > >>>>>> script,  
> > > >> > > > >>>>>> >and option lists for both cases (Intel and gcc) for  
> > any  
> > > >> > > reference.  
> > > >> > > > >>>>>> >
> > > >> > > > >>>>>> >Secondly, we are not allowed to compile ETK on login  
> > nodes.  
> > > >> So  
> > > >> > > we are  
> > > >> > > > >>>>>> >queuing a job script, which upon execution, enters the  
> > > >> Cactus  
> > > >> > > > >>>>>> directory,  
> > > >> > > > >>>>>> >does "setup-silent", and then executes the build  
> > command.  
> > > >> Is  
> > > >> > > this  
> > > >> > > > >>>>>> the right  
> > > >> > > > >>>>>> >way to compile ETK using the batch script? Or should we  
> > do  
> > > >> > > something  
> > > >> > > > >>>>>> >different? I have attached the shell script  
> > (compile.sh)  
> > > >> used  
> > > >> > > for  
> > > >> > > > >>>>>> sbatch  
> > > >> > > > >>>>>> >queue for reference.  
> > > >> > > > >>>>>>
> > > >> > > > >>>>>> I might be wrong, but setup-silent will not use your  
> > machine  
> > > >> file  
> > > >> > > > >>>>>> which
> > > >> > > > >>>>>> contains all the compiler flags, but will setup  
> > something  
> > > >> "from  
> > > >> > > > >>>>>> scratch"
> > > >> > > > >>>>>> instead. According to the logs, this will be, e.g., in
> > > >> > > > >>>>>>
> > > >> > > > >>>>>>
> > > >> > > > >>>>>>  
> > > >> > >  
> > > >>  
> > /home/shamims.iiserb/ET_debug/Cactus/repos/simfactory2/mdb/machines/cn139.iitr.ac.in.ini  
> > > >>  
> > > >> > > > >>>>>>
> > > >> > > > >>>>>> My guess would be that this misses the compiler flags  
> > > >> necessary to  
> > > >> > > > >>>>>> have
> > > >> > > > >>>>>> M_PI defined. The best way to get this working might be  
> > to  
> > > >> let  
> > > >> > > > >>>>>> simfactory also detect the cluster configuration on the  
> > > >> compute  
> > > >> > > nodes  
> > > >> > > > >>>>>> (where you compile), such that setup-silent is not  
> > needed,  
> > > >> but  
> > > >> > > > >>>>>> instead
> > > >> > > > >>>>>> the correct cluster configuration is found and used  
> > > >> > > automatically. In  
> > > >> > > > >>>>>> order to do that, look at examples of 'aliaspattern' in  
> > the  
> > > >> > > > >>>>>> mdb/machines
> > > >> > > > >>>>>> directory. An alternative would be to tell simfactory  
> > > >> specifically  
> > > >> > > > >>>>>> that
> > > >> > > > >>>>>> you want to use your machine configuration file.
> > > >> > > > >>>>>>
> > > >> > > > >>>>>> As a side-note: I noticed the command 'patch' is  
> > missing  
> > > >> too.  
> > > >> > > This is  
> > > >> > > > >>>>>> a
> > > >> > > > >>>>>> tool so common that it should be installed everywhere.  
> > Your  
> > > >> admin  
> > > >> > > is  
> > > >> > > > >>>>>> probably the best person to ask for advise here. It  
> > should  
> > > >> not be  
> > > >> > > too  
> > > >> > > > >>>>>> hard to install yourself, but that should not be  
> > necessary  
> > > >> either,  
> > > >> > > > >>>>>> especially when you are told to compile on compute nodes.
> > > >> > > > >>>>>>
> > > >> > > > >>>>>> Frank
> > > >> > > > >>>>>>
> > > >> > > > >>>>>>  
> > > >> > > > >>>>> _______________________________________________
> > > >> > > > >>>>> Users mailing listUsers at einsteintoolkit.orghttp://  
> > > >> > > lists.einsteintoolkit.org/mailman/listinfo/users  
> > > >> > > > >>>>>
> > > >> > > > >>>>> _______________________________________________
> > > >> > > > >>>>> Users mailing list
> > > >> > > > >>>>> Users at einsteintoolkit.org
> > > >> > > > >>>>>  
> > > >> > >  
> > > >>  
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!4xAKSrBYUo_v8jHEbenNg6aQ99Oq9QyeXsjq766lVDVwHAhFJODYjSU6XryKp264bQqDmBcTIA4Srkc4wOCn$  
> > > >>  
> > > >> > > > >>>>>  
> > > >> > > > >>>>  
> > > >> > >
> > > >> > >
> > > >> > > Yours,
> > > >> > > Roland
> > > >> > >
> > > >> > > --
> > > >> > > My email is as private as my paper mail. I therefore support  
> > > >> encrypting  
> > > >> > > and signing email messages. Get my PGP key from  
> > > >>  
> > https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!5ZGhPGL_16Uj1eN8xbeLFN1po4t2MOETlRoEnO3fr4pBXNdsn_VXBbuywp1tgk0zKYy_oBKphC1Ny-KY0Vh9$  
> > > >> .  
> > > >> > >  
> > > >>
> > > >> --
> > > >> My email is as private as my paper mail. I therefore support  
> > encrypting  
> > > >> and signing email messages. Get my PGP key from  
> > https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!9Rc70AgC5_rUJApjF6bdaoR4iJf6WXHxF1UsHWThqJwM8QxTwHKDWcVgn0kphw4Lmahcfk5a8gMIPSpV8TMD$
> > .  
> > > >>  
> > > >  
> >
> >
> > --
> > My email is as private as my paper mail. I therefore support encrypting
> > and signing email messages. Get my PGP key from https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!4Z697XBz_Q8JIQSrOykhIqb_thU0tH0XLyTUT1zDMwhZlk1earaaKDJXweGCrJ3LGb2HvWAHbeRBlS57nxFF$  .
> >  


Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230606/46430976/attachment-0001.sig>


More information about the Users mailing list