#1775: Add Boost to ET
Reporter: Erik Schnetter
Status: new
Milestone: ET_2026_05
Version: development version
Type: enhancement
Priority: minor
Component: EinsteinToolkit thorn
Comment (by Roland Haas):
Note that my Debian box (and self-compiled Boost) do have a libboost_system:
```
/usr/lib/x86_64-linux-gnu/libboost_system.a libboost-system1.83-dev [amd64], libboost-system1.88-dev [amd64]
/usr/lib/x86_64-linux-gnu/libboost_system.so libboost-system1.83-dev [amd64], libboost-system1.88-dev [amd64]
```
where did you encounter that file not being present?
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/1775/add-boost-to-et
#1775: Add Boost to ET
Reporter: Erik Schnetter
Status: new
Milestone: ET_2026_05
Version: development version
Type: enhancement
Priority: minor
Component: EinsteinToolkit thorn
Comment (by Roland Haas):
In the ET call we collected some boost sub-libraries that are used in the ET:
* system and filesystem (since in David's original code)
* math (or so) by CarpetX's Algo library
* ptree (FUKA), Python (FUKA) -- won't provide this since it opens up a can of worms trying to build for Python
* optional, format, math, odeint, (RrPrimAnd, just says "Boost", but this is what is actually required to compile it)
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/1775/add-boost-to-et
#1775: Add Boost to ET
Reporter: Erik Schnetter
Status: new
Milestone: ET_2026_05
Version: development version
Type: enhancement
Priority: minor
Component: EinsteinToolkit thorn
Comment (by Roland Haas):
oh, it does not like `-j` alone. Grr. Apparently I coded for this (there's special code) but never tested it.
Fixed. Also removed the check for a `system` linker library.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/1775/add-boost-to-et
#1775: Add Boost to ET
Reporter: Erik Schnetter
Status: new
Milestone: ET_2026_05
Version: development version
Type: enhancement
Priority: minor
Component: EinsteinToolkit thorn
Comment (by Roland Haas):
hmm, would be nice if it told me what that invalid value was. Could you re-run with `export VERBOSE=yes`, please? That should show you the shell command that the build script tries to execute and thus the bad option value as well.
it really only ever should be 1. a number (incl. 1), 2. nothing (if make was called with `-j` without any options).
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/1775/add-boost-to-et
#1775: Add Boost to ET
Reporter: Erik Schnetter
Status: new
Milestone: ET_2026_05
Version: development version
Type: enhancement
Priority: minor
Component: EinsteinToolkit thorn
Comment (by Peter Diener):
When trying the thorn in the repository, I got this output:
Boost: Configuring...
Building B2 engine..
###
###
### Using 'gcc' toolset.
###
###
g++ (GCC) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
###
###
> g++ -x c++ -std=c++11 -O2 -s -DNDEBUG builtins.cpp class.cpp command.cpp compile.cpp constants.cpp cwd.cpp debug.cpp debugger.cpp execcmd.cpp execnt.cpp execunix.cpp filesys.cpp filent.cpp fileunix.cpp frames.cpp function.cpp glob.cpp hash.cpp hcache.cpp hdrmacro.cpp headers.cpp jam_strings.cpp jam.cpp jamgram.cpp lists.cpp make.cpp make1.cpp md5.cpp mem.cpp modules.cpp native.cpp object.cpp option.cpp output.cpp parse.cpp pathnt.cpp pathsys.cpp pathunix.cpp regexp.cpp rules.cpp scan.cpp search.cpp startup.cpp subst.cpp sysinfo.cpp timestamp.cpp variable.cpp w32_getreg.cpp modules/order.cpp modules/path.cpp modules/property-set.cpp modules/regex.cpp modules/sequence.cpp modules/set.cpp -o b2
tools/build/src/engine/b2
Detecting Python version... 2.7
Detecting Python root... /usr
Unicode/ICU support for Boost.Regex?... /usr
Generating B2 configuration in project-config.jam for gcc...
Bootstrapping is done. To build, run:
./b2
To generate header files, run:
./b2 headers
The configuration generated uses gcc to build by default. If that is
unintended either use the --with-toolset option or adjust configuration, by
editing 'project-config.jam'.
Further information:
- Command line help:
./b2 --help
- Getting started guide:
http://www.boost.org/more/getting_started/unix-variants.html
- B2 documentation:
http://www.boost.org/build/
Boost: Building...
Invalid value for the '-j' option.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/1775/add-boost-to-et
#6: Inconsistent computation of the volume form in Coordinates (llama) between Thornburg04/13 and default behavior
Reporter: Jordan Nicoules
Status: submitted
Milestone:
Version:
Type: bug
Priority: major
Component:
See ticket:
https://bitbucket.org/einsteintoolkit/tickets/issues/2912/inconsistent-comp…
**IMPORTANT WARNING: THIS IS A BEHAVIOR CHANGE AND MAY BREAK BACKWARD COMPATIBILITY FOR NON-THORNBURG04/13 RUNS!**
# Modifications to implement: #
## Coordinates/src/inverse_jacobian.F90 ##
See attached file.
### WARNING ###
This was tested only on Thornburg04 and Thornburg04nc.
Thornburg13 works like Thornburg04 and redefines its volume form, so it should be fine.
For other patch systems, I'm not sure if this is safe for each subpatch, but I think that shouldn't be more wrong than the current implementation.
### NOTE ###
Feel free to change the comments I made, and/or adapt the existing TODO in comment (lines 59-61) if it's relevant.
## Coordinates/src/thornburg04.cc
Fix typo in comment line 1439:
`// set volume form to determinant of Jacobian`
## Coordinates/src/thornburg13.cc
Fix typo in comment line 2237:
`// set volume form to determinant of Jacobian`
attachment: inverse_jacobian.F90 (https://api.bitbucket.org/2.0/repositories/llamacode/llama/issues/6/attachm…)
--
Ticket URL: https://bitbucket.org/llamacode/llama/issues/6/inconsistent-computation-of-…
#2930: Bug in QuasiLocalMeasures
Reporter: Marco Brito
Status: new
Milestone:
Version:
Type: bug
Priority: major
Component: EinsteinToolkit thorn
I found a bug in the QuasiLocalMeasures thorn, in the computation of the ADM momentum. In the source code file qlm_analyse.F90, in line 361 there is a plus sign instead of a minus sign, contradicting the ADM momentum formula: P^i_{ADM} = \frac{1}{8\pi}\lim_{r\to\infty} \oint_S (K^i_j - \delta^i_j K)dS^j [1]
As expected for slices where K \neq 0 one was getting a wrong result. The correction is easy, just replace the plus sign by a minus sign. You can find the corrected file in attachment.
[1] M. Alcubierre, Introduction to 3+1 Numerical Relativity, International Series of Monographs on Physics (Oxford University Press, Oxford, 2008).
attachment: qlm_analyse.F90 (https://api.bitbucket.org/2.0/repositories/einsteintoolkit/tickets/issues/2…)
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2930/bug-in-quasilocal…
#2929: GRHayLET/IllinoisGRMHD convert_IllinoisGRMHD_to_HydroBase schedule compatibility with VolumeIntegrals_*
Reporter: Maxwell Rizzo
Status: new
Milestone: ET_2026_05
Version: ET_2025_05
Type: bug
Priority: major
Component: EinsteinToolkit thorn
Comment (by Roland Haas):
As a suggestion: there is a Cactus parameter
```
Cactus::schedule_sort_mode = "descending"
```
also "non", the default, or "ascending".
Which can be used to willfully change the ordering of scheduled functions without any explicit `AFTER` or `BEFORE` relation. A correctly written `schedule.ccl` should be unaffected. While one that relies on accidental order would fail.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2929/grhaylet-illinois…
#2929: GRHayLET/IllinoisGRMHD convert_IllinoisGRMHD_to_HydroBase schedule compatibility with VolumeIntegrals_*
Reporter: Maxwell Rizzo
Status: new
Milestone: ET_2026_05
Version: ET_2025_05
Type: bug
Priority: major
Component: EinsteinToolkit thorn
Comment (by Leonardo Rosa Werneck):
Thanks for pointing this out!
I created [PR #15](https://github.com/GRHayL/GRHayLET/pull/15) to address this issue. Will post an update once it's reviewed and merged.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2929/grhaylet-illinois…
#2923: ADIOS2 compilation fails due to unknown type name ‘INT4’
Reporter: Cheng-Hsin Cheng
Status: new
Milestone:
Version: ET_2025_05
Type: bug
Priority: major
Component: EinsteinToolkit thorn
Comment (by Steven R. Brandt):
Discussed on the ET call.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2923/adios2-compilatio…
#2909: Remove TAGS from interface.ccl
Reporter: Steven R. Brandt
Status: open
Milestone: ET_2025_05
Version: ET_2025_05
Type: enhancement
Priority: major
Component: Cactus
Comment (by Steven R. Brandt):
Discussed on today's ET call.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2909/remove-tags-from-…
#2929: GRHayLET/IllinoisGRMHD convert_IllinoisGRMHD_to_HydroBase schedule compatibility with VolumeIntegrals_*
Reporter: Maxwell Rizzo
Status: new
Milestone: ET_2026_05
Version: ET_2025_05
Type: bug
Priority: major
Component: EinsteinToolkit thorn
In GRHayLET/IllinoisGRMHD/schedule.ccl, starting at line 220, it appears that the Convert_to_Hydrobase is scheduled before the wrong thorn VolumeIntegral_* schedule group.
```
if(Convert_to_HydroBase_every) {
schedule convert_IllinoisGRMHD_to_HydroBase at CCTK_ANALYSIS before (compute_bi_b2_Poyn_fluxET convert_to_MHD_3velocity particle_tracerET VolumeIntegralGroup) after ML_BSSN_evolCalcGroup
{
LANG: C
OPTIONS: GLOBAL-EARLY,LOOP-LOCAL
READS: ADMBase::metric, ADMBase::lapse, ADMBase::shift,
grmhd_velocities, grmhd_B_center
WRITES: HydroBase::vel(everywhere), HydroBase::w_lorentz(everywhere), HydroBase::Bvec(everywhere)
} "Convert IllinoisGRMHD-native variables to HydroBase"
}
```
VolumeIntegralGroup, appearing above in the ```before``` list, is the schedule group for thorn VolumeIntegrals_vacuum which does not inherit HydroBase. VI_GRMHD_VolumeIntegralGroup is the schedule group for thorn VolumeIntegrals_GRMHD which does inherit and use HydroBase.
The above schedule does not ensure conversion to HydroBase occurs before VolumeIntegrals_GRMHD runs its main scheduled group.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2929/grhaylet-illinois…
#1775: Add Boost to ET
Reporter: Erik Schnetter
Status: new
Milestone: ET_2026_05
Version: development version
Type: enhancement
Priority: minor
Component: EinsteinToolkit thorn
Comment (by Roland Haas):
I note that while this builds a whole lot of libraries, it only, matching David's old library, links in `system` and `filesystem`. So we could likely speed up the build process by passing `--with-system` and `--with-filesystem` to `b2`.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/1775/add-boost-to-et
#2928: New Simfactory Option
Reporter: Steven R. Brandt
Status: new
Milestone:
Version:
Type: enhancement
Priority: major
Component: SimFactory
The options to simfactory reflect usage on older hardware. We used to be more concerned with how many cores we wanted to use and how many to allocate to each process. Now we are often more concerned with how many gpus we want. It would be much more convenient to say something like:
```
./simfactory/bin/sim create-run --gpus=4 --queue=gpu2 ...
```
Than to figure out that nodes in the gpu2 queue we have 64 cores and 2 gpus per node, and thus the way to invoke simfactory is as follows:
```
./simfactory/bin/sim create-run --procs=128 --num-threads=32 ...
```
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2928/new-simfactory-op…
#1775: Add Boost to ET
Reporter: Erik Schnetter
Status: new
Milestone: ET_2026_05
Version: development version
Type: enhancement
Priority: minor
Component: EinsteinToolkit thorn
Comment (by Roland Haas):
It is possible to interact with make's parallel build system (somewhat): https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html (note that Cactus requires GNU make).
```
#!/bin/bash
# fail on errors, error on unset variable reference
set -u -e
# verobse output?
if [ ${VERBOSE-no} = yes ]; then
set -x
fi
# handle -n option to make (do nothing), since make thinks we are a sub-make
set -- $MAKEFLAGS
if [[ ${#*} -ge 1 ]] && [[ $1 = *n* ]]; then # -n flag passed to make, do nothing
exit 0
fi
# parse remaining make arguments
MAX_JOBS=1
JOBSERVER_AUTH=
for o in "$@"; do
case $o in
( --jobserver-auth=* )
JOBSERVER_AUTH=${o#--jobserver-auth=}
;;
( -j* )
MAX_JOBS=${o#-j*}
;;
( -- )
break
;;
esac
done
# functions to get some job tokens from make
function return_tokens() {
local JOBSERVER_FILE=
if [ -n "$JOBSERVER_AUTH" ] && [ -n "$JOBSERVER_TOKENS" ]; then
case $JOBSERVER_AUTH in
( fifo:* )
JOBSERVER_FILE=${JOBSERVER_AUTH#fifo:}
;;
( *,* )
# /dev/fd/ emulated by bash (if not by the OS)
JOBSERVER_FILE=/dev/fd/${JOBSERVER_AUTH#*,} # read handle of pipe
;;
esac
if [ -n "$JOBSERVER_FILE" ]; then
echo -n $JOBSERVER_TOKENS >$JOBSERVER_FILE
fi
fi
}
function maybe_get_tokens() {
local JOBSERVER_FILE=
if [ -n "$JOBSERVER_AUTH" ]; then
case $JOBSERVER_AUTH in
( fifo:* )
JOBSERVER_FILE=${JOBSERVER_AUTH#fifo:}
;;
( *,* )
# /dev/fd/ emulated by bash (if not by the OS)
JOBSERVER_FILE=/dev/fd/${JOBSERVER_AUTH#%,*} # write handle of pipe
;;
esac
if [ -n "$JOBSERVER_FILE" ]; then
set +e # read sets error code on timeout, which is not an error
read -r -N $MAX_JOBS -t 5 JOBSERVER_TOKENS <$JOBSERVER_FILE
set -e
fi
fi
}
if [ -n "$MAX_JOBS" ]; then
JOBSERVER_TOKENS=
# wait for 5 seconds to get up MAX_JOBS tokens (or 1024 if MAX_JOBS is empty)
trap return_tokens EXIT
maybe_get_tokens
# one job is for "free" since it represents this process
JOBS_OPT=-j$(( ${#JOBSERVER_TOKENS} + 1 ))
else # no maximum
JOBS_OPT=-j
fi
echo $JOBS_OPT tokens: ${JOBSERVER_TOKENS:-}
```
when declaring the script above a sub-make to make using:
```
all:
+@./bjam-test.sh
```
which then produces:
```
haengie2: ~/.../Boost/dist$ make -j5 -f GNUMakefile all
-j5 tokens: ++++
haengie2: ~/.../Boost/dist$ make -j1 -f GNUMakefile all
-j1 tokens:
haengie2: ~/.../Boost/dist$ make -j -f GNUMakefile all
-j tokens:
haengie2: ~/.../Boost/dist$ make -f GNUMakefile all
-j1 tokens:
```
ie constructs a `-j` option for bjam reflection (a subset of) the jobs available to make.
As for tar file size, the best I can do right now is to remove Boost examples, docs, tests etc. then re-compress with `gzip`'s `--rsyncable` option for a ~30MB file size which git can hopefully diff:
```
#!/bin/bash
# this script removes "extra" files from Boost distribution archive to try and
# reduce its size. It creates a new zipped archive that may be diff-able for git.
if [ ${#@} -ne 1 ]; then
echo >&2 "usage: $0 <tar-file>"
exit 1
fi
set -e
FN="$1"
TEMPDIR=`mktemp -d`
function cleanup() {
rm -r $TEMPDIR
}
trap cleanup EXIT
tar -xf "$FN" -C $TEMPDIR
find $TEMPDIR -depth '(' -name examples -or -name doc -or -name test ')' -print0 | xargs --null rm -r
# use gzip's --rsyncable option in hopes that this will let git diff versions
# of the tar archive
( cd $TEMPDIR ; tar -c * ) | gzip --rsyncable >${FN%.tar*}-stripped.tar.gz
```
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/1775/add-boost-to-et
#2927: gcc-version-dependent error: Range error setting parameter
Reporter: Jordan Nicoules
Status: new
Milestone:
Version:
Type: bug
Priority: major
Component: Cactus
For reference, thread on the mailing list: https://lists.einsteintoolkit.org/pipermail/users/2026-March/009901.html
# TLDR #
With certain versions of gcc, having a (real) parameter defined with a range only consisting of ``*`` causes a range error at the start of the simulation.
# Description #
With certain versions of gcc (13.3.0, 14.3.0), having a real parameter defined with a range only consisting of ``*`` causes a range error at the start of the simulation, right after thorn activation:
```
WARNING[L2,P0] (Cactus): ParameterSetReal: Unable to set real 'Coordinates::h_radial_1' = '0.40000000000000002' not in any active range
WARNING[L1,P0] (Cactus): Major error in parameter file '/home/jnicoules/Work/ET_test/Simus/parfiles/Gallery/Kerr-Schild_Multipole_stretch.par' line 46: Range error setting parameter 'Coordinates::h_radial_1' to '0.40000000000000002'
WARNING level 0 from host cnx001.deucalion.macc.fccn.pt process 0
in thorn Cactus, file /projects/F202407872CPCAA2/jnicoules/ET_test/Cactus/configs/ET_test__x86__GCC-13.3.0/build/Cactus/main/ProcessParameterDatabase.c:201:
-> CCTKi_SetParameterSetMask: 1 major error in parameter file
```
The issue was encountered with a specific parameter, ``Coordinates::h_radial_1``, from the ``llama`` repo. I didn't experiment with other parameters from different thorns.
Most of the experimentation was performed on x86 nodes of the Deucalion cluster (https://docs.macc.fccn.pt/deucalion/). Three different versions of gcc and available related modules were tested: 12.3.0, 13.3.0, 14.3.0.
# Minimal (non)working example #
The issue can be seen with the attached parameter file, based on the Multipatch wave equation Gallery example, tweaked to include parameter ``Coordinates::h_radial_1``. The used ET version for the MWE is 2025-05 and the thornlist is a shaved off version of the one on the ET website (https://bitbucket.org/einsteintoolkit/manifest/raw/ET_2025_05/einsteintoolk…), to ease configuration and compilation.
The outputs are attached, for each of the three versions mentioned above. Additionally, it contains the output for a configuration using GCC-13.3.0, but changing, in ``Cactus/repos/llama/Coordinates/param.ccl``,
```
real h_radial_1 "Intended radial resolution of the first stretched domain"
{
* :: "negative turns off stretching"
} -1
```
by
```
real h_radial_1 "Intended radial resolution of the first stretched domain"
{
*:* :: "negative turns off stretching"
} -1
```
which is enough to have the simulation work properly.
# Other notes #
As mentioned in https://lists.einsteintoolkit.org/pipermail/users/2026-March/009901.html, gcc 15 might work just well.
Testing on my other setups (workstation and other clusters), where I have various older versions of gcc, the issue did not occur. I'm not attaching the respective files here, as the MWE shows both working and non-working situations.
# Attachments #
```
range_error_MWE/
|__ einstein_toolkit.th # Thornlist
|__ Kerr-Schild_Multipole_stretch.par # Parameter file
|__ Kerr-Schild_Multipole_stretch/ # Contains the output for the various cases, without .h5 files for size reasons
|__ README
|__ output-*/
|__ GCC-*/ # contains compilation information in each case
|__ *.cfg # option file
|__ config_*.log # output of make config
|__ make_*.log # output of make
|__ make.config.defn # file found in Cactus/configs/*/config-data
```
attachment: range_error_MWE.zip (https://api.bitbucket.org/2.0/repositories/einsteintoolkit/tickets/issues/2…)
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2927/gcc-version-depen…