Hello José,

Thanks for the detailed report.

I do not really use conda myself unfortunately, so will not be able to
directly test it though (unless things really go sideways).

The error you are encountering now is that the linker cannot find the
libz.so (or libz.a or libz.la) linker file. This is the zlib library
used for compression by eg HDF5. 

This can have multiple reasons. During configuration Cactus (really the
ExternalLibraries/zlib thorn) tries to determine where zlib is located
and set up linker options accordingly. It does so by (among other
things) searching "well known" locations for the file (among them /usr
and others). If found in one of those "well known" locations, which are
normally included in the compiler's default search path, it will not
add an explicit linker option so as to not move a location from the
"default" priority to "high" priority via an -L option (since this
causes no end of issues, and is eg also what cmake does).

Long story short, the anaconda provided linker may not look at those
"well known" locations since it wants to keep everything contained in
the anaconda environment (which mostly works for Python but not really
for anything else). 

To fix this you'd do like you already do for eg GSL and add a

ZLIB_DIR = /home/undercover/.micromamba/envs/phd/

setting to your option list (assuming that say libz.so is in /home/undercover/.micromamba/envs/phd/lib/libz.so).

Similarly for other errors of the same sort.

The automated search and build functionality in Cactus is mostly geared
towards vanilla Linux installations using one of the distributions
listed in
https://github.com/einsteintoolkit/jupyter-et/blob/master/tutorial-server/notebooks/CactusTutorial.ipynb
in the "Prerequisites" sections or macOS (using HomeBrew or MacPorts).

When used in other circumstances (on clusters, or, apparently, with
[mini]conda) failures are unfortunately common and one has to provide much
more information manually. Hopefully the suggestion above helps a bit.

Coincidentally I presented a talk on the Cactus build system (and some
ways of verifying what is going on) at the ET summer school and
workshop last week at LSU, which may help you find out exactly what is
going on. You can find the slides for this here (and soon also on the
summer school web-page):

https://github.com/rhaas80/2024_LSU/blob/main/CactusBuildSytemTour.ipynb

Yours,
Roland

On Wed, 12 Jun 2024 19:04:49 +0100, José Ferreira wrote:

Hello Roland,


Thanks for the reply, and apologies for not including all relevant
information.

One of the things is that I forgot to mention that conda (in this
case, micromamba, a drop-in alternative that is fully compatible)
sets some environmental variables by itself. Obviously, some of them
are are the overwritten with whatever is on the |.cfg| file.

It turns out that micromamba sets |$LD| to point towards the |ld|
binary provided by the virtual environment

Adding |LD = g++| as you suggested resulted in a slightly different
error

|Creating cactus_empty in /home/undercover/projects/cactus/exe from
/home/undercover/.micromamba/envs/phd/bin/../lib/gcc/x86_64-conda-linux-gnu
/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find -lz:
No such file or directory collect2: error: ld returned 1 exit status
make[1]: ***
[/home/undercover/projects/cactus/lib/make/make.configuration:150:
/home/undercover/projects/cactus/exe/cactus_empty] Error 1 make: ***
[Makefile:265: empty] Error 2 |


To avoid back and forth between e-mails, and to add some information
that was previously missing, here are the steps that I take to
reproduce the error

1.

Create a new conda environment and activate it (in my machine it__
_s
called “phd”)

2.

|conda install gcc gxx gfortran openmpi openmpi-mpicc
openmpi-mpicxx openmpi-mpifort binutils|

3.

Compile the toolkit with the thornlist |empty.th| and the options
file |venv.cfg|, both attached again to this e-mail

4.

Configuration is created without any issues, with the terminal
output attached to |empty.log| (this file was missing in the
previous e-mail), and get the error making the binary, with the
terminal output attached in |cactus_empty.log|


Further relevant information about the running status of the machine:

*

Environmental variables: see file |env.txt| which includes a
stripped down version of the environmental variables. For
instance, you can see that |$LD| is set and carries on with its value
during compilation, but |$LDFLAGS|, |$CFLAGS| and |$CXXFLAGS| are
overwritten by whatever is in the |venv.cfg|. Even if I
concatenate the environmental variables from the shell with the ones
from the options file, it still doesn’t work.

*

OS: Manjaro

*

Kernel: Linux legion 5.10.218-1-MANJARO #1 SMP PREEMPT Mon May 27
02:19:19 UTC 2024 x86_64 GNU/Linux

*

CPU: Intel(R) Core(TM) i5-7300HQ CPU @ 2.50GHz (quad core)

*

Package Manager: Micromamba
<https://urldefense.com/v3/__https://mamba.readthedocs.io/en/latest/use
r_guide/micromamba.html__;!!DZ3fjg!4JUhc_Pyghv7QqhhZMN_0qShBOabzxJWmOQed8oS
me19pKFVPvO_FgbMz2l69QVIuUf2kUS14LoJ0DHZJOeg$ >
using the conda-forge
<https://urldefense.com/v3/__https://conda-forge.
org/packages/__;!!DZ3fjg!4JUhc_Pyghv7QqhhZMN_0qShBOabzxJWmOQed8oSme19pKFVPv
O_FgbMz2l69QVIuUf2kUS14LoJ0BBzkfxl$ > repositories


It should be noted that I am able of replicating this bug on my
laptop (this machine) and two different desktops that are both
running Arch Linux.

This is not totally unexpected considering they are very similar OS__
_s with the same versions of the software that is being used.


I hope that I could make it as clear as possible.

Best regards,

José Ferreira


On 12/06/24 15:59, Roland Haas wrote:

Hello Jose,

Cactus uses the C++ compiler to link things.

This is controlled by the LD option list option.

In your option list (thank you for including it) you do not set it,
so it should default to the value of CXX.

However if there is an environment variable LD set, then this will
override the option list default (you'd see eg LD in the output of
the env command).

So as a first try, I'd add

LD = g++

to your option list and recompile from scratch.

If that does not help, then more information will be required.
Please see:

https://urldefense.com/v3/__http://einsteintoolkit.org/support.html*gener

al-guidelines-for-questions__;Iw!!DZ3fjg!4JUhc_Pyghv7QqhhZMN_0qShBOabzxJWmO
Qed8oSme19pKFVPvO_FgbMz2l69QVIuUf2kUS14LoJ0A8OitkU$ >

for what to include.

Yours,
Roland

On Wed, 12 Jun 2024 15:51:13 +0100, José Ferreira wrote:  

Dear toolkit users and developers,


With the latest update of GCC to version 14, there is a known error
that prevents the compilation from taking place, preventing me from
compiling the toolkit.

I’ve been trying to do something which I probably should have so  

me

time ago to avoid this kind of issues, while also granting some
portability between machines, which is to setup a virtual
environment using Conda to compile the toolkit.


I am sending you this e-mail because I’ve failed to compile the
toolkit in a virtual environment, where I am currently stuck with
an error during the linking phase. Before the update to GCC 14, and
without the virtual environment, I was able of compiling the
toolkit without any issue.

Here are the steps that I have performed:

1.

Activate an empty virtual environment managed by a conda (or a
drop-in replacement) with the conda-forge repository configured

2.

|conda install gcc gxx gfortran openmpi openmpi-mpicc
openmpi-mpicxx openmpi-mpifort binutils|

3.

Compile the toolkit with an empty thornlist, attached in
|empty.th|, and an option file that points towards the virtual
environment that is activated, attached in |venv.cfg|

4.

Get the following error from |ld| (the full log for making the
config and the binary are attached in |cactus_empty.log|)

|Creating cactus_empty in /home/undercover/projects/cactus/exe from
/home/undercover/.micromamba/envs/phd/bin/x86_64-conda-linux-gnu-ld:
unrecognised emulation mode: arch=native Supported emulations:
elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu make[1]: ***
[/home/undercover/projects/cactus/lib/make/make.configuration:150:
/home/undercover/projects/cactus/exe/cactus_empty] Error 1 make:
*** [Makefile:265: empty] Error 2 |


To me, the previous error indicates that something is passing the
wrong flags to |ld|, because even if the emulation mode was not
recognized it should give the error |ld: unrecognised emulation
mode: <something>| instead of |ld: unrecognised emulation mode:
arch=<something>|.

Removing the flag |-march=native| from the compilers confirms my
guess by revealing a different error at the same stage

|Creating cactus_empty in /home/undercover/projects/cactus/exe from
/home/undercover/.micromamba/envs/phd/bin/x86_64-conda-linux-gnu-ld:
unrecognized option '-DMPICH_IGNORE_CXX_SEEK'
/home/undercover/.micromamba/envs/phd/bin/x86_64-conda-linux-gnu-ld:
use the --help option for usage information make[1]: ***
[/home/undercover/projects/cactus/lib/make/make.configuration:150:
/home/undercover/projects/cactus/exe/cactus_empty] Error 1 make:
*** [Makefile:265: empty] Error 2 |

and if I were to remove the corresponding flag in the config file,
|ld| will complain about something else.

It really seems like something is passing the flags onto |ld| where
it shouldn’t, but I cannot tell who and where.


I should note that compiling very basic programs in C in the
virtual environment doesn’t raise any issues, although I haven’t  

 tested it

very thoroughly.

Please do let me know if you have any issues in rendering this
e-mail!


Best regards,

José Ferreira


​  

​