Hello, I am facing an issue where running a simulation with the toolkit, using the default thornlist, can either run as expected, or crash right in the beginning of the simulation. This took place with ET_TOV.par, that ships with the toolkit, and a thornfile that evolves a constant scalar field, that ships with the Scalar thorn also included in the toolkit.

I believe that the culprit is OpenMP because if I disable threading during run-time, by setting OMP_NUM_THREADS=1, or if I disable OpenMP during compile-time, the simulations run as expected.

This bug takes place both in my laptop and in my desktop, which share similar operating systems and software stack.

‌

In the following sections, I will write, line by line, the steps that I have performed in order to reproduce the bug, and my attempt of tracking it down.

‌

Installing and Compiling the Toolkit

To avoid compiling CarpetX thorns that fail to compile in my system, for some reason, I start by downloading the previous version of the toolkit with

$ curl -kLO https://raw.githubusercontent.com/gridaphobe/CRL/ET_2023_05/GetComponents
$ chmod a+x GetComponents
$ ./GetComponents --parallel https://bitbucket.org/einsteintoolkit/manifest/raw/ET_2023_05/einsteintoolkit.th

and then change to the Cactus root directory

$ cd Cactus

I create the options file arch.cfg , which should already be present in the parent directory, with the following written in it

# Cactus configuration for Arch and Arch-based
attachment:
backtrace.0.txt
attachment:
make-binary.out
attachment:
make-binary-debug.out
attachment:
make-config.out
attachment:
make-config-debug.out
attachment:
run.out
attachment:
run.sh
attachment:
run-debug.out

--

Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2774/openmp-heisenbug-with-default-thornfile