Hi Roland,

In the config file I attached, I do have this line included already:

FPPFLAGS = -g -traditional

Is there anything else I might try?

Best, Panayotis ________________________________ From: Roland Haas rhaas@phas.ubc.ca Sent: Tuesday, September 2, 2025 5:08 PM To: IOSIF PANAGIOTIS PANAGIOTIS.IOSIF@units.it Cc: Einstein Toolkit Users users@einsteintoolkit.org Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster)

Hello Panayotis,

Thanks for including all the output files.

In your options list whenever your set FPP you also must also set FPPFLAGS:

FPPFLAGS=-traditional

See eg generic.cfg or (very new!) Bruno's leonardo-DCGP.cfg file (in simfactory master).

Yours, Roland

[CAUTION: Non-UBC Email]

Hi Einstein Toolkit users,

I am reaching out regarding some errors I am encountering during my attempts to compile ET on the Leonardo, CINECA cluster (DCGP partition).

Here are some details on the Leonardo cluster:

cluster specifics: https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo-card

OS: Red Hat Enterprise Linux 8.7

I attach the 4 configuration files I am using (with many added comments for guidance and future reference). I adapted them from the respective config files that Bruno Giacomazzo kindly provided.

I also attach the make.log output of the command:

./simfactory/bin/sim build --verbose --reconfig --machine=leonardo-dcgp1 --optionlist simfactory/mdb/optionlists/leonardo-dcgp1.cfg --allocation=CNHPC_1479290_0 --thornlist thornlists/einsteintoolkit.th 2>&1 | tee make.log

From what I understand, the build process complains about syntax errors in the Exact thorn in these two source files:

decode_pars.F

Boost_rotation_symmetric.F

How can I bypass the above problems?

To be honest, I am not sure that I really need this thorn. I was simply trying to compile ET using the default thornlist from a fresh download/installation.

Is there any other error that the log file is pointing out that I might be missing?

Best regards, Panayotis

---

Panagiotis Iosif postdoctoral researcher Department of Physics, University of Trieste Via Alfonso Valerio 2, Trieste 34127 Italy

---

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

Hi Roland,

I amended my cfg file and added the flag -ffixed-line-length-none , in the Fortran 77 and 90 compiler options:

* F77FLAGS = -g -march=native -fcray-pointer -ffixed-line-length-none * F90FLAGS = -g -march=native -fcray-pointer -ffixed-line-length-none

This step solved the issue with the complaints about the "Exact" thorn.

The build process continued and stopped at a different point, now complaining about the CarpetX/Arith thorn.

I have attached the new make.log file.

I ran the log through ChatGPT, and it suggested that this could be caused by C++ standard mismatches. Does this sound reasonable to you?

Currently, the flags for the C and C++ compilers in my config file are like this:

* CFLAGS = -g -march=native -std=gnu99 * CXXFLAGS = -g -march=native -std=gnu++11

I checked Bruno's file, and the respective options there are the following:

* CFLAGS = -g -std=gnu99 #-std=c11 * CXXFLAGS = -g -std=gnu++0x #-std=c++14

However, I should note that in Bruno's version, all CarpetX-related thorns are disabled in the ini file.

Again, I am not sure I need CarpetX. At this point, I am trying to get the build process to work and run some examples on the cluster.

Returning to the compiler options, I suspect that CFLAGS can be left alone. Do you have any suggestions on how to change CXXFLAGS to bypass the CarpetX/Arith complaints?

I checked the different C dialect options herehttps://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html, and it seems that there are a number of possibilities.

Is there some -std=... option that is new enough for CarpetX and old/standard enough so that older C++ code is also OK?

Best, Panayotis

________________________________ From: Roland Haas rhaas@phas.ubc.ca Sent: Tuesday, September 2, 2025 5:43 PM To: IOSIF PANAGIOTIS PANAGIOTIS.IOSIF@units.it Cc: Einstein Toolkit Users users@einsteintoolkit.org Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster)

Hello Panayotis,

Oh, I had not noticed, sorry. Hmm, that is the typical cause for these issues.

The only other difference I see is `-ffixed-line-length-none` though that would also point to an issue with the Cactus Fortran file processing not breaking things into short enough lines.

Yours, Roland

[CAUTION: Non-UBC Email]

Hi Roland,

In the config file I attached, I do have this line included already:

FPPFLAGS = -g -traditional

Is there anything else I might try?

Best, Panayotis ________________________________ From: Roland Haas rhaas@phas.ubc.ca Sent: Tuesday, September 2, 2025 5:08 PM To: IOSIF PANAGIOTIS PANAGIOTIS.IOSIF@units.it Cc: Einstein Toolkit Users users@einsteintoolkit.org Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster)

Hello Panayotis,

Thanks for including all the output files.

In your options list whenever your set FPP you also must also set FPPFLAGS:

FPPFLAGS=-traditional

See eg generic.cfg or (very new!) Bruno's leonardo-DCGP.cfg file (in simfactory master).

Yours, Roland

...

[CAUTION: Non-UBC Email]

Hi Einstein Toolkit users,

I am reaching out regarding some errors I am encountering during my attempts to compile ET on the Leonardo, CINECA cluster (DCGP partition).

Here are some details on the Leonardo cluster:

cluster specifics: https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo-card

OS: Red Hat Enterprise Linux 8.7

I attach the 4 configuration files I am using (with many added comments for guidance and future reference). I adapted them from the respective config files that Bruno Giacomazzo kindly provided.

I also attach the make.log output of the command:

./simfactory/bin/sim build --verbose --reconfig --machine=leonardo-dcgp1 --optionlist simfactory/mdb/optionlists/leonardo-dcgp1.cfg --allocation=CNHPC_1479290_0 --thornlist thornlists/einsteintoolkit.th 2>&1 | tee make.log

From what I understand, the build process complains about syntax errors in the Exact thorn in these two source files:

decode_pars.F

Boost_rotation_symmetric.F

How can I bypass the above problems?

To be honest, I am not sure that I really need this thorn. I was simply trying to compile ET using the default thornlist from a fresh download/installation.

Is there any other error that the log file is pointing out that I might be missing?

Best regards, Panayotis

---

Panagiotis Iosif postdoctoral researcher Department of Physics, University of Trieste Via Alfonso Valerio 2, Trieste 34127 Italy

---

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

Hi Roland,

Got it.

Here is an update:

I removed all CarpetX-related thorns and tried to compile again. This time, the build complained about thorns GRHayLHDX, GRHayLIDX, and NewRadX:

CST error 1: -> Thorn 'GRHayLHDX' requires the capability 'Loop'. Please add a thorn that provides 'Loop' to your ThornList or remove 'GRHayLHDX' from it !

CST error 2: -> Thorn 'GRHayLIDX' requires the capability 'Loop'. Please add a thorn that provides 'Loop' to your ThornList or remove 'GRHayLIDX' from it !

CST error 3: -> Thorn 'NewRadX' requires the capability 'Loop'. Please add a thorn that provides 'Loop' to your ThornList or remove 'NewRadX' from it !

Comparing with Bruno's original file, I saw that he had disabled GRHayLHDX and GRHayLIDX. So I went ahead and disabled these two thorns for starters.

Then the build complained just about the NewRadX thorn (same error message as above).

So I disabled NewRadX too (see updated .ini file attached) and the error about NewRadX went away.

Now the build complains about the CCE_Export thorn. You may see the error messages in the updated log file attached.

From what I understand, the first main error seems to be this (see make_updated.log file):

/leonardo/home/userexternal/piosif00/Cactus/arrangements/EinsteinAnalysis/CCE_Export/src/h5_export.cc:8:21: error: 'filesystem' is not a namespace-name; did you mean 'system'? 8 | namespace fs = std::filesystem; The text around that error message suggests that again a newer C++ dialect option might be required (-std=c++17' or '-std=gnu++17' ).

In my current cfg file, following what was mentioned in the wiki page about configuring a new machinehttps://docs.einsteintoolkit.org/et-docs/Configuring_a_new_machine, I have an older option, namely -std=gnu+11, for CXXFLAGS.

I will experiment with different C standards options and see if something else works.

Questions:

* Are GRHayLHDX, GRHayLIDX and NewRadX safe to disable? * For context, my goal is to start with isolated neutron star simulations. Are these thorns necessary for that? * Do we expect newer C/C++ standards, like -std=c++17, to break backwards compatibility, i.e. older code? * Please let me know if I am on the right track, or if you see some additional issues from the log file.

Let me note that I did not start with all of the disabled thorns that Bruno's original ini filehttps://bitbucket.org/simfactory/simfactory2/src/a6dff6ecc4a4346c5a73536681486e745d34bb98/mdb/machines/leonardo-DCGP.ini uses, because I didn't know exactly what was needed or not. Currently, the disabled thorns in my ini file almost align with those in Bruno's original file. There are, though, some additional thorns disabled in Bruno's original file, namely: ADIOS2, AMReX, Silo, PAPI and THCExtra/WeakRates.

I do not know if I should disable them too, but since I did not encounter specific errors about them, I let them be. Let me know if they should go too.

Thanks for the help so far!

Best, Panayotis

________________________________ From: Roland Haas rhaas@phas.ubc.ca Sent: Tuesday, September 2, 2025 7:05 PM To: IOSIF PANAGIOTIS PANAGIOTIS.IOSIF@units.it Cc: rhaas@mail.ubc.ca rhaas@mail.ubc.ca; Einstein Toolkit Users users@einsteintoolkit.org Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster)

Hello Panayotis,

oh, oops, for CarpetX (which includes Arith) code you actually need -std=c++17 .

If you are not planning to develop for CarpetX then you can remove all CarpetX related thorns from your thornlist.

Note that if you have a specific parameter file that you would like to run, then you can use the MakeThornList helper script to create a thornlist with just the thorns required for that parameter file, potentially speeding up compilation quite a bit.

Something like:

./utils/Scripts/MakeThornList --master thornlists/einsteintoolkit.th --output thornlists/qc0.th par/qc0-mclachlan.par

if you wanted to run qc0-mclachlan.par and had the full thornlist in thornlists/einsteintoolkit.th (GetComponents will do that for you).

Then pass qc0.th to simfactory using its --thornlist argument (you likely also need to pass --reconfig to make sure it actually updates the thornlist it uses).

Yours, Roland

[CAUTION: Non-UBC Email]

Hi Roland,

I amended my cfg file and added the flag -ffixed-line-length-none , in the Fortran 77 and 90 compiler options:

F77FLAGS = -g -march=native -fcray-pointer -ffixed-line-length-none

F90FLAGS = -g -march=native -fcray-pointer -ffixed-line-length-none

This step solved the issue with the complaints about the "Exact" thorn.

The build process continued and stopped at a different point, now complaining about the CarpetX/Arith thorn.

I have attached the new make.log file.

I ran the log through ChatGPT, and it suggested that this could be caused by C++ standard mismatches. Does this sound reasonable to you?

Currently, the flags for the C and C++ compilers in my config file are like this:

CFLAGS = -g -march=native -std=gnu99

CXXFLAGS = -g -march=native -std=gnu++11

I checked Bruno's file, and the respective options there are the following:

CFLAGS = -g -std=gnu99 #-std=c11

CXXFLAGS = -g -std=gnu++0x #-std=c++14

However, I should note that in Bruno's version, all CarpetX-related thorns are disabled in the ini file.

Again, I am not sure I need CarpetX. At this point, I am trying to get the build process to work and run some examples on the cluster.

Returning to the compiler options, I suspect that CFLAGS can be left alone. Do you have any suggestions on how to change CXXFLAGS to bypass the CarpetX/Arith complaints?

I checked the different C dialect options herehttps://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html, and it seems that there are a number of possibilities.

Is there some -std=... option that is new enough for CarpetX and old/standard enough so that older C++ code is also OK?

Best, Panayotis

---

From: Roland Haas rhaas@phas.ubc.ca Sent: Tuesday, September 2, 2025 5:43 PM To: IOSIF PANAGIOTIS PANAGIOTIS.IOSIF@units.it Cc: Einstein Toolkit Users users@einsteintoolkit.org Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster)

Hello Panayotis,

Oh, I had not noticed, sorry. Hmm, that is the typical cause for these issues.

The only other difference I see is `-ffixed-line-length-none` though that would also point to an issue with the Cactus Fortran file processing not breaking things into short enough lines.

Yours, Roland

...

[CAUTION: Non-UBC Email]

Hi Roland,

In the config file I attached, I do have this line included already:

FPPFLAGS = -g -traditional

Is there anything else I might try?

Best, Panayotis ________________________________ From: Roland Haas rhaas@phas.ubc.ca Sent: Tuesday, September 2, 2025 5:08 PM To: IOSIF PANAGIOTIS PANAGIOTIS.IOSIF@units.it Cc: Einstein Toolkit Users users@einsteintoolkit.org Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster)

Hello Panayotis,

Thanks for including all the output files.

In your options list whenever your set FPP you also must also set FPPFLAGS:

FPPFLAGS=-traditional

See eg generic.cfg or (very new!) Bruno's leonardo-DCGP.cfg file (in simfactory master).

Yours, Roland

...

[CAUTION: Non-UBC Email]

Hi Einstein Toolkit users,

I am reaching out regarding some errors I am encountering during my attempts to compile ET on the Leonardo, CINECA cluster (DCGP partition).

Here are some details on the Leonardo cluster:

cluster specifics: https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo-card * OS: Red Hat Enterprise Linux 8.7

I attach the 4 configuration files I am using (with many added comments for guidance and future reference). I adapted them from the respective config files that Bruno Giacomazzo kindly provided.

I also attach the make.log output of the command:

./simfactory/bin/sim build --verbose --reconfig --machine=leonardo-dcgp1 --optionlist simfactory/mdb/optionlists/leonardo-dcgp1.cfg --allocation=CNHPC_1479290_0 --thornlist thornlists/einsteintoolkit.th 2>&1 | tee make.log

From what I understand, the build process complains about syntax errors in the Exact thorn in these two source files:

decode_pars.F

Boost_rotation_symmetric.F

How can I bypass the above problems?

To be honest, I am not sure that I really need this thorn. I was simply trying to compile ET using the default thornlist from a fresh download/installation.

Is there any other error that the log file is pointing out that I might be missing?

Best regards, Panayotis

---

Panagiotis Iosif postdoctoral researcher Department of Physics, University of Trieste Via Alfonso Valerio 2, Trieste 34127 Italy

---

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

Yours, Roland

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

Hi Roland,

Thank you very much for your detailed explanations.

Here is another follow-up that seems to have a happy ending:

* I disabled the ADIOS2, AMReX and Silo thorns and recompiled. * I did not add THCExtra/WeakRates to the disabled thorn list. *

From what I understand, this must probably be related to WhiskyTHC, which I do not have installed.

* The build complained about the CMake version, triggered by the ExternalLibraries/OpenPMD thorn.

openPMD: Configuring... CMake Error at CMakeLists.txt:3 (cmake_minimum_required): CMake 3.22.0 or higher is required. You are running version 3.20.2

* For some reason, the build this time did not complain about the CCE_Export thorn, even though I did not update the -std=gnu+11 option under CXXFLAGS yet (I was trying to make one change at a time for clarity). * I found this behavior strange. I guess the CMake error happened first, and the build terminated before reaching the CCE_Export problem?

Anyway, this error seemed easy enough to fix.

* I updated envsetup in my ini file with: * module load cmake/3.27.9 * Recompiled, and the CMake version error went away, but (expectedly) the CCE_Export error reappeared.

Then, I tried recompiling by adding -std=gnu++14 to CXXFLAGS.

* The build failed, complaining again about the CCE_Export thorn. * At least the change from gnu+11 to gnu++14did not cause new additional errors. * at least no new errors, in the specific cluster and with the specific config files.

Then I tried recompiling by adding -std=gnu++17 to CXXFLAGS.

*

From the attached log file, it seems to me that the build was successful this time.

* No new errors arose because of the change to -std=gnu++17 in CXXFLAGS.

I will now attempt to run the TOV star example from the gallery and see if everything works as it should.

I will start a new email thread in case I encounter run problems.

I appreciate the help!

Best, Panayotis

------------------- Panagiotis Iosif postdoctoral researcher Department of Physics, University of Trieste Via Alfonso Valerio 2, Trieste 34127 Italy -------------------

________________________________ From: Roland Haas rhaas@mail.ubc.ca Sent: Wednesday, September 3, 2025 5:29 PM To: IOSIF PANAGIOTIS PANAGIOTIS.IOSIF@units.it Cc: Einstein Toolkit Users users@einsteintoolkit.org Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster)

Hello Panayotis,

I removed all CarpetX-related thorns and tried to compile again. This time, the build complained about thorns GRHayLHDX, GRHayLIDX, and NewRadX:

They are all using CarpetX (the "X" is the giveaway :-) ).

Now the build complains about the CCE_Export thorn.

Hmm, that one is very new.

You may see the error messages in the updated log file attached.

From what I understand, the first main error seems to be this (see make_updated.log file): /leonardo/home/userexternal/piosif00/Cactus/arrangements/EinsteinAnalysis/CCE_Export/src/h5_export.cc:8:21: error: 'filesystem' is not a namespace-name; did you mean 'system'? 8 | namespace fs = std::filesystem; The text around that error message suggests that again a newer C++ dialect option might be required (-std=c++17' or '-std=gnu++17' ).

Hmm, for C++'s filesystem namespace is somewhat new. Some older compiler put it in experimental/filesystem . However CCE_Export already has code to handle this (in src/h5_export.cc):

#if defined __cpp_lib_filesystem && __cpp_lib_filesystem < 201703L #include <experimental/filesystem> namespace fs = std::experimental::filesystem; #else #include <filesystem> namespace fs = std::filesystem; #endif

Hmm, you are using gcc-12 though I'd have expected it to be new enough for this.

Hmm, hmm, those __cpp_lib_filesystem ones may only haven been introduced in C++20 (https://en.cppreference.com/w/cpp/feature_test.html) and they require the <version> header, which itself is C++20 (though it exists in gcc-12 and will have values).

Unfortunately even if I add

#include <version>

to src/h5_export.cc then things still fail with -std=gnu++11 since the macros are only defined when C++17 is used.

So.... I'd disable the thorn (only used to export data for use with SpECTRE's CCE code) or enable C++17 support.

This may require some larger reworking of the ET code to try and contain C++17 requirements to CarpetX code if possible.

In my current cfg file, following what was mentioned in the wiki page about configuring a new machinehttps://docs.einsteintoolkit.org/et-docs/Configuring_a_new_machine, I have an older option, namely -std=gnu+11, for CXXFLAGS.

gnu+11 is kind of old by now, at least I'd try gnu++14 (10 years old). So no guarantees that this would work (gcc defaults to gnu++17 as of version 11 "C++17 mode is the default since GCC 11" on https://gcc.gnu.org/projects/cxx-status.html).

I will experiment with different C standards options and see if something else works.

I'd have expected that both gnu++14 and gnu++17 should work actually (as long as CarpetX related code is disable since it and eg AMReX require C++17).

Questions:

Are GRHayLHDX, GRHayLIDX and NewRadX safe to disable?

Yes, they are all using CarpetX.

 *

For context, my goal is to start with isolated neutron star simulations. Are these thorns necessary for that?

No, they are CarpetX flavors of the GRHayLHD, GRHayLID and NewRad thorns.

Do we expect newer C/C++ standards, like -std=c++17, to break backwards compatibility, i.e. older code?

Yes, new standards will eventually remove functionality that has been deprecated in older ones. On top of that g++ is becoming more strict in allowing non-standard constructs. This is, unfortunately, beyond our control.

Please let me know if I am on the right track, or if you see some additional issues from the log file.

Let me note that I did not start with all of the disabled thorns that Bruno's original ini filehttps://bitbucket.org/simfactory/simfactory2/src/a6dff6ecc4a4346c5a73536681486e745d34bb98/mdb/machines/leonardo-DCGP.ini uses, because I didn't know exactly what was needed or not. Currently, the disabled thorns in my ini file almost align with those in Bruno's original file. There are, though, some additional thorns disabled in Bruno's original file, namely: ADIOS2, AMReX, Silo, PAPI and THCExtra/WeakRates.

All except PAPI (which you'll only need if doing low-level benchmarking) are using only by CarpetX (or in WeakRates case are no even part of the Einstein Toolkit), so those can also be safely disabled.

I do not know if I should disable them too, but since I did not encounter specific errors about them, I let them be. Let me know if they should go too.

It will make compiling faster (note the MakeThornList script I had suggested will also remove them).

Yours, Roland

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .