[Users] New Testing Framework for Individual Einstein Toolkit Thorns

[Zach Etienne]

Einstein Toolkit community:

I have completed a new, automated, and fast framework for testing
individual or small groups of thorns within the Einstein Toolkit. Although
it is developed with GitHub Actions in mind, it can be run locally. As I
believe this may be useful for the community, I am sharing all the details
in this email.

While the framework is developed for thorns generated with NRPy2, it can be
simplified fairly straightforwardly for thorns developed by hand. Here's
how it works:

With each push to the NRPy2 `main` branch, the following happens (as
documented here:
https://github.com/nrpy/nrpy/blob/main/.github/workflows/main.yml#L136):
GitHub Actions first downloads an Apptainer (formerly Singularity) SIF
image I have prepared here:
https://github.com/nrpy/einsteintoolkit_ci_images/releases. The above image
contains a recent (24.04) Ubuntu distribution with two pieces of software
installed in /opt:
1. A *compiled* version of a recent (pre-2024-06 release) version of the
Einstein Toolkit, with Baikal*, IllinoisGRMHD, WaveToyNRPy, and all other
required thorns.
2. A recent NRPy2.

After downloading the above image, Actions runs a script that pulls the
latest NRPy2 main branch, generates Baikal* and WaveToyNRPy thorns,
overwrites these thorns in the Einstein Toolkit, and then compiles them
within the Toolkit. Since other thorns are not compiled, this is a very
quick check to ensure that updates to NRPy2-generated thorns do not break
the compilation.

Finally, the Einstein Toolkit testsuite is run on each of these thorns. If
any of the above steps ({download, thorn generation, compilation,
testsuite}) fail, then GitHub Actions fails and it automatically sends me
an unpleasant email.

We plan to extend the framework to CarpetX (Baikal*X and WaveToyNRPyX
thorns) in the coming months, probably with a separate Apptainer image.
Please feel free to use the Apptainer image available at
https://github.com/nrpy/einsteintoolkit_ci_images/releases, or the script
available at
https://github.com/nrpy/nrpy/blob/main/.github/workflows/main.yml#L136 if
you may find it useful. Everything here is open source.

This complements the very nice existing Einstein Toolkit testing framework (
https://einsteintoolkit.github.io/tests/), which builds the entire Toolkit
automatically after any ET thorn or the flesh is updated. The full build
requires about 90 minutes, whereas my testing framework is more limited in
scope and completes in under 10 minutes (as it focuses only on a small
collection of thorns).

-Zach

*     *     *
Zachariah Etienne
Assoc. Prof. of Physics, U. of Idaho
Adjunct Assoc. Prof. of Physics & Astronomy, West Virginia U.
https://etienneresearch.com
https://blackholesathome.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20240618/9ea63158/attachment.htm>