Dear Roland,
Thanks for the prompt response.
You solution got rid of the error, so it does seem to be a path issues after all.
However, right after that, I got another error
|/home/undercover/.micromamba/envs/phd/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/undercover/.micromamba/envs/phd/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../lib/libopen-pal.so: undefined reference to `memcpy@GLIBC_2.14' /home/undercover/.micromamba/envs/phd/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/undercover/.micromamba/envs/phd/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../lib/./libpmix.so.2: undefined reference to `clock_gettime@GLIBC_2.17' /home/undercover/.micromamba/envs/phd/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/undercover/.micromamba/envs/phd/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../lib/./././libicuuc.so.73: undefined reference to `std::condition_variable::wait(std::unique_lockstd::mutex&)@GLIBCXX_3.4.30' collect2: error: ld returned 1 exit status |
which seems to be related to GLIBC.
From a quick search online, it seems that conda does not allow from a custom version of glibc, and the packages available in conda-forge use the system glibc, as can be seen when inside the virtual environment
|$ ldd $(which gcc) linux-vdso.so.1 (0x00007ffd03a67000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f134b3a3000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f134b1b7000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f134b4ca000) |
My guess would be that GCC expects a given version of GLIBC (more than one, apparently, which feels weird to me) and is unable to find them on my system, because I cannot provide them.
My understanding on these topics is sparse at best, so I'm just going to believe that there is no way to insert a custom glibc here somehow.
To be clear, I’m not longer asking for your support on this, I’m merely just reporting it for you and those that happen to be following this thread.
Thank you for your assistance,
All the best,
José Ferreira
*P.S.:* Is there any workaround on https://bitbucket.org/einsteintoolkit/tickets/issues/2802/compilation-faiulr..., for instance, by inserting some flag on GCC 14 and recover the warning where now there is an error?
On 12/06/24 19:29, Roland Haas wrote:
Hello José,
Thanks for the detailed report.
I do not really use conda myself unfortunately, so will not be able to directly test it though (unless things really go sideways).
The error you are encountering now is that the linker cannot find the libz.so (or libz.a or libz.la) linker file. This is the zlib library used for compression by eg HDF5.
This can have multiple reasons. During configuration Cactus (really the ExternalLibraries/zlib thorn) tries to determine where zlib is located and set up linker options accordingly. It does so by (among other things) searching "well known" locations for the file (among them /usr and others). If found in one of those "well known" locations, which are normally included in the compiler's default search path, it will not add an explicit linker option so as to not move a location from the "default" priority to "high" priority via an -L option (since this causes no end of issues, and is eg also what cmake does).
Long story short, the anaconda provided linker may not look at those "well known" locations since it wants to keep everything contained in the anaconda environment (which mostly works for Python but not really for anything else).
To fix this you'd do like you already do for eg GSL and add a
ZLIB_DIR = /home/undercover/.micromamba/envs/phd/
setting to your option list (assuming that say libz.so is in /home/undercover/.micromamba/envs/phd/lib/libz.so).
Similarly for other errors of the same sort.
The automated search and build functionality in Cactus is mostly geared towards vanilla Linux installations using one of the distributions listed in https://github.com/einsteintoolkit/jupyter-et/blob/master/tutorial-server/no... in the "Prerequisites" sections or macOS (using HomeBrew or MacPorts).
When used in other circumstances (on clusters, or, apparently, with [mini]conda) failures are unfortunately common and one has to provide much more information manually. Hopefully the suggestion above helps a bit.
Coincidentally I presented a talk on the Cactus build system (and some ways of verifying what is going on) at the ET summer school and workshop last week at LSU, which may help you find out exactly what is going on. You can find the slides for this here (and soon also on the summer school web-page):
https://github.com/rhaas80/2024_LSU/blob/main/CactusBuildSytemTour.ipynb
Yours, Roland
On Wed, 12 Jun 2024 19:04:49 +0100, José Ferreira wrote:
Hello Roland,
Thanks for the reply, and apologies for not including all relevant information.
One of the things is that I forgot to mention that conda (in this case, micromamba, a drop-in alternative that is fully compatible) sets some environmental variables by itself. Obviously, some of them are are the overwritten with whatever is on the |.cfg| file.
It turns out that micromamba sets |$LD| to point towards the |ld| binary provided by the virtual environment
Adding |LD = g++| as you suggested resulted in a slightly different error
|Creating cactus_empty in /home/undercover/projects/cactus/exe from /home/undercover/.micromamba/envs/phd/bin/../lib/gcc/x86_64-conda-linux-gnu /11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find -lz: No such file or directory collect2: error: ld returned 1 exit status make[1]: *** [/home/undercover/projects/cactus/lib/make/make.configuration:150: /home/undercover/projects/cactus/exe/cactus_empty] Error 1 make: *** [Makefile:265: empty] Error 2 |
To avoid back and forth between e-mails, and to add some information that was previously missing, here are the steps that I take to reproduce the error
Create a new conda environment and activate it (in my machine it__ _s called “phd”)
|conda install gcc gxx gfortran openmpi openmpi-mpicc openmpi-mpicxx openmpi-mpifort binutils|
Compile the toolkit with the thornlist |empty.th| and the options file |venv.cfg|, both attached again to this e-mail
Configuration is created without any issues, with the terminal output attached to |empty.log| (this file was missing in the previous e-mail), and get the error making the binary, with the terminal output attached in |cactus_empty.log|
Further relevant information about the running status of the machine:
Environmental variables: see file |env.txt| which includes a stripped down version of the environmental variables. For instance, you can see that |$LD| is set and carries on with its value during compilation, but |$LDFLAGS|, |$CFLAGS| and |$CXXFLAGS| are overwritten by whatever is in the |venv.cfg|. Even if I concatenate the environmental variables from the shell with the ones from the options file, it still doesn’t work.
OS: Manjaro
Kernel: Linux legion 5.10.218-1-MANJARO #1 SMP PREEMPT Mon May 27 02:19:19 UTC 2024 x86_64 GNU/Linux
CPU: Intel(R) Core(TM) i5-7300HQ CPU @ 2.50GHz (quad core)
Package Manager: Micromamba <https://urldefense.com/v3/__https://mamba.readthedocs.io/en/latest/use r_guide/micromamba.html__;!!DZ3fjg!4JUhc_Pyghv7QqhhZMN_0qShBOabzxJWmOQed8oS me19pKFVPvO_FgbMz2l69QVIuUf2kUS14LoJ0DHZJOeg$ > using the conda-forge <https://urldefense.com/v3/__https://conda-forge. org/packages/__;!!DZ3fjg!4JUhc_Pyghv7QqhhZMN_0qShBOabzxJWmOQed8oSme19pKFVPv O_FgbMz2l69QVIuUf2kUS14LoJ0BBzkfxl$ > repositories
It should be noted that I am able of replicating this bug on my laptop (this machine) and two different desktops that are both running Arch Linux.
This is not totally unexpected considering they are very similar OS__ _s with the same versions of the software that is being used.
I hope that I could make it as clear as possible.
Best regards,
José Ferreira
On 12/06/24 15:59, Roland Haas wrote:
Hello Jose,
Cactus uses the C++ compiler to link things.
This is controlled by the LD option list option.
In your option list (thank you for including it) you do not set it, so it should default to the value of CXX.
However if there is an environment variable LD set, then this will override the option list default (you'd see eg LD in the output of the env command).
So as a first try, I'd add
LD = g++
to your option list and recompile from scratch.
If that does not help, then more information will be required. Please see:
https://urldefense.com/v3/__http://einsteintoolkit.org/support.html*gener
al-guidelines-for-questions__;Iw!!DZ3fjg!4JUhc_Pyghv7QqhhZMN_0qShBOabzxJWmO Qed8oSme19pKFVPvO_FgbMz2l69QVIuUf2kUS14LoJ0A8OitkU$ >
for what to include.
Yours, Roland
On Wed, 12 Jun 2024 15:51:13 +0100, José Ferreira wrote:
Dear toolkit users and developers,
With the latest update of GCC to version 14, there is a known error that prevents the compilation from taking place, preventing me from compiling the toolkit.
I’ve been trying to do something which I probably should have so
me
time ago to avoid this kind of issues, while also granting some portability between machines, which is to setup a virtual environment using Conda to compile the toolkit.
I am sending you this e-mail because I’ve failed to compile the toolkit in a virtual environment, where I am currently stuck with an error during the linking phase. Before the update to GCC 14, and without the virtual environment, I was able of compiling the toolkit without any issue.
Here are the steps that I have performed:
Activate an empty virtual environment managed by a conda (or a drop-in replacement) with the conda-forge repository configured
|conda install gcc gxx gfortran openmpi openmpi-mpicc openmpi-mpicxx openmpi-mpifort binutils|
Compile the toolkit with an empty thornlist, attached in |empty.th|, and an option file that points towards the virtual environment that is activated, attached in |venv.cfg|
Get the following error from |ld| (the full log for making the config and the binary are attached in |cactus_empty.log|)
|Creating cactus_empty in /home/undercover/projects/cactus/exe from /home/undercover/.micromamba/envs/phd/bin/x86_64-conda-linux-gnu-ld: unrecognised emulation mode: arch=native Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu make[1]: *** [/home/undercover/projects/cactus/lib/make/make.configuration:150: /home/undercover/projects/cactus/exe/cactus_empty] Error 1 make: *** [Makefile:265: empty] Error 2 |
To me, the previous error indicates that something is passing the wrong flags to |ld|, because even if the emulation mode was not recognized it should give the error |ld: unrecognised emulation mode: <something>| instead of |ld: unrecognised emulation mode: arch=<something>|.
Removing the flag |-march=native| from the compilers confirms my guess by revealing a different error at the same stage
|Creating cactus_empty in /home/undercover/projects/cactus/exe from /home/undercover/.micromamba/envs/phd/bin/x86_64-conda-linux-gnu-ld: unrecognized option '-DMPICH_IGNORE_CXX_SEEK' /home/undercover/.micromamba/envs/phd/bin/x86_64-conda-linux-gnu-ld: use the --help option for usage information make[1]: *** [/home/undercover/projects/cactus/lib/make/make.configuration:150: /home/undercover/projects/cactus/exe/cactus_empty] Error 1 make: *** [Makefile:265: empty] Error 2 |
and if I were to remove the corresponding flag in the config file, |ld| will complain about something else.
It really seems like something is passing the flags onto |ld| where it shouldn’t, but I cannot tell who and where.
I should note that compiling very basic programs in C in the virtual environment doesn’t raise any issues, although I haven’t
tested it
very thoroughly.
Please do let me know if you have any issues in rendering this e-mail!
Best regards,
José Ferreira