From jnicoules at ua.pt  Wed Apr  1 10:04:22 2026
From: jnicoules at ua.pt (Jordan Nicoules)
Date: Wed, 1 Apr 2026 15:04:22 +0000
Subject: [Users] Range error setting parameter, on cluster
In-Reply-To: <20260318091801.3ab29488@haengie2.phas.ubc.ca>
References: <3ff153d1eb844753a4f1ca29e61183b0@ua.pt>
 <20260212071459.7aa0e964@haengie2.phas.ubc.ca>
 <e7808c34078f4b8ab8403d6eb4228229@ua.pt>
 <20260226070049.5fdb5113@haengie2.phas.ubc.ca>
 <7bd4ec7b04ef4ee9912b4e997af7c7da@ua.pt>
 <20260227073009.5cd6b294@haengie2.phas.ubc.ca>
 <711db3ef25944b5fa55bb73f1de72f1e@ua.pt>
 <20260312073303.2ff22ef2@haengie2.phas.ubc.ca>
 <7b336dc2027b4ff8b5e7c7043cdd20e3@ua.pt>,
 <20260318091801.3ab29488@haengie2.phas.ubc.ca>
Message-ID: <d275f3517a784028ab2d2e735be85af7@ua.pt>

Hi Roland,


I have just created a new ticket to report the issue more cleanly. I included a minimal (non-)working example and configuration related information, that hopefully should help reproduce the issue. Sorry I couldn't do it sooner. Unfortunately, I probably won't be able to join the weekly meeting tomorrow.


Best,


Jordan

________________________________
From: Roland Haas <rhaas at mail.ubc.ca>
Sent: Wednesday, March 18, 2026 4:18:01 PM
To: Jordan Nicoules
Cc: Jordan Nicoules via Users
Subject: Re: [Users] Range error setting parameter, on cluster


CUIDADO: Email de um sistema externo. Cuidado com links, anexos e pedidos de dados/senhas.
CAUTION: Email from an external system. Be careful with links, attachments, and requests for data/passwords.

Hello Jordan,

Adding this to the ticket would be great.

On my workstation I have gcc-15 and things work well there, so this may
be only some specific version that is affected, so knowing the exact
number is helpful (one could even compile that version and test).

Yours,
Roland

> [CAUTION: Non-UBC Email]
>
> Hi Roland (and Peter),
>
>
> Thank you for the suggestion! This was very relevant.
>
> With loaded modules related to GCC-13.3.0 (and GCC-14.3.0), the error is occurring, while for GCC-12.3.0, the range is working as expected. This is compatible with the experiment on other clusters and on my work station, which have lower versions of gcc as well.
>
>
> What should be made of that knowledge? Should I file a ticket? I suppose that in the future, this may be ever more likely to occur. (For my individual case, it's fine, I can deal with it.)
>
>
> Best,
>
>
> Jordan
>
> ________________________________
> From: Roland Haas <rhaas at mail.ubc.ca>
> Sent: Thursday, March 12, 2026 2:33:03 PM
> To: Jordan Nicoules via Users
> Cc: Jordan Nicoules
> Subject: Re: [Users] Range error setting parameter, on cluster
>
>
> CUIDADO: Email de um sistema externo. Cuidado com links, anexos e pedidos de dados/senhas.
> CAUTION: Email from an external system. Be careful with links, attachments, and requests for data/passwords.
>
> Hello Jordan,
>
> We discussed this some more in today's ET call and Peter Diener
> suggested trying a different compiler on Deucalian just in case this is
> actually a compiler issue.
>
> Yours,
> Roland
>
> > [CAUTION: Non-UBC Email]
> >
> > Hi Roland,
> >
> >
> > Indeed, Deucalion is a cluster (as well as MesoPSL and MareNostrum, which I checked before sending my previous email, and which seem to work just fine with the "*" range as expected).
> >
> > I'm attaching the requested files:
> >
> > - for the work station: relayer_ubuntu.cfg, workstation_make.config.defn
> >
> > - for the Deucalion cluster: deucalion-x86.cfg, deucalion_make.config.defn
> >
> >
> > These are from the minimal working example I was mentioning before, with the latest ET version.
> >
> >
> > Thank you for your assistance!
> >
> >
> > Best,
> >
> >
> > Jordan
> >
> > ________________________________
> > From: Roland Haas <rhaas at mail.ubc.ca>
> > Sent: Friday, February 27, 2026 3:30:09 PM
> > To: Jordan Nicoules
> > Cc: Jordan Nicoules via Users
> > Subject: Re: [Users] Range error setting parameter, on cluster
> >
> >
> > CUIDADO: Email de um sistema externo. Cuidado com links, anexos e pedidos de dados/senhas.
> > CAUTION: Email from an external system. Be careful with links, attachments, and requests for data/passwords.
> >
> > Hello Jordan,
> >
> > > Well, I don't know if the git hash corresponds to the ET release directly (I took it from the front page of the UsersGuide.pdf I have on my work station). The corresponding ET release should be ET_2024_05 I think.
> >
> > The hash that is shown in the docs is a git commit hash of the Cactus
> > repository (the "flesh") so I can correlate with the release.
> >
> > > I'm not sure this is (completely) a matter of version, since for a given version, I get different behaviors on different machines:
> > >
> > > - ET_2024_05: Deucalion --> range error ; work station --> fine
> > >
> > > - ET_2025_05: Deucalion --> range error ; work station, MesoPSL, MareNostrum --> all fine.
> > >
> >
> > Ok, it working on a workstation and failing on another (or cluster,
> > https://en.wikipedia.org/wiki/Deucalion_(supercomputer) ?) is very strange.
> >
> > Would you be able to send me the option lists for those two and also
> > the files configs/<FOO>/config-data/make.config.defn ? The latter are
> > the fully parsed files that make constructs using the option list and
> > other information.
> >
> > > Isn't ET_2025_05 the latest?
> >
> > It is the latest yes.
> >
> > Yours,
> > Roland
> >
> > --
> > My email is as private as my paper mail. I therefore support encrypting
> > and signing email messages. Get my PGP key from http://pgp.mit.edu .
>
>
> --
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://pgp.mit.edu .

--
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20260401/690d3190/attachment.htm>

From users at einsteintoolkit.org  Wed Apr  1 17:15:01 2026
From: users at einsteintoolkit.org (users at einsteintoolkit.org)
Date: Wed, 01 Apr 2026 17:15:01 -0500
Subject: [Users] Einstein Toolkit Meeting Reminder
Message-ID: <69cd98e5.1ZfxN36zddV3lc5t%users@einsteintoolkit.org>

Hello,

Please consider joining the weekly Einstein Toolkit phone call at
9:00 am US central time on Thursdays. For details on how to connect
and what agenda items are to be discussed, use the link below.

https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call

--The Maintainers

From sbrandt at cct.lsu.edu  Thu Apr  2 09:57:51 2026
From: sbrandt at cct.lsu.edu (Steven Brandt)
Date: Thu, 2 Apr 2026 09:57:51 -0500
Subject: [Users] Meeting minutes for 2026-04-02
Message-ID: <fec42fef-1e6e-400a-8f3c-01e62b348515@cct.lsu.edu>

April 2
Present: Peter, Steve, Roland, Lucas, Deborah, Beyhan, Maxwell Rizzo, 
Cheng-Hsin, Zach, Nikolai Wallin
Chair: Deborah
Minutes: Steve

Release seems to be on track
The TOV Gallery is updated
 ? ? Previous one is too big because too much was uploaded
Need to link BH-NS collision to the gallery page
TwopuncturesX is having trouble
Cheng-Hsin says the Teukolsky-Lean test--fails on 2 procs, but not on 2 
threads
 ? ? Is output written in consistent order?
Cottonmouth - Split into 5 parts to obtain optimal, Zach asks whether 
these can run in parallel?
Zach says that makes the point that higher order often needs different 
splittings
Peter asked about the CPU performance
CanudaX review, Lucas has started, needs doc folder (as does Cottonmouth).
BHaHAHA review, Peter says it has a lot of good documentation
New Gallery--all done, ready to go
Boost--Roland has a script to reduce the tarball to 30M or so.
 ? ?Boost has its own build system, bjam. Hooked into make's parallel 
build system.
 ? ?Considered splitting it into multiple tarballs.
Newest boost doesn't have boost::math::tools::eps_tolerance, so can't 
use it.

Workshop and School
 ? ? SOC meeting this afternoon

Atlassian retiring bitbucket issues thingy
 ? ? Zach will move that in May

Kyoto initiative comments? Who should we ask for updates?
 ? ? Zach, Philip Moesta, Beyhan, and Jay are involved.
 ? ? Zach is happy to be the Kyoto initiative liason.

Unanswered emails:
 ? ? Range error email, new issue created, using a different compiler 
made it go away

Tickets:
 ? ?ADIOS2 - Cheng-Hsin - looks at the build logs, int is size 4. On 
Delta. Has had the same issue on the local Ubuntu laptop.
 ? ?2909 - Reminder to Roland.

"A question that sometimes drives me hazy: am I or are the others 
crazy?" --Einstein


From nikolaiwallin at gmail.com  Thu Apr  2 09:46:48 2026
From: nikolaiwallin at gmail.com (Nikolai Wallin)
Date: Thu, 2 Apr 2026 07:46:48 -0700
Subject: [Users] Meeting minutes for 2026-03-26
Message-ID: <CAHKMMxLigv5MWkpHqqzCp7eo+=mtjm6GxUsi=Z_5=-ywPGP-GA@mail.gmail.com>

Release:
-Cottonmouth in master, Doc directory is non existent.
This needs to be fixed. Currently the fastest according to Steve.
-Two Punctures X should be added. Currently added to master
-Probably need to update the CI system to ensure numpy is installed.
Currently failing
-Overall release is making progress,
we need to get the reviews out of the way.
-Need to add some work to the website front page.
-Peter is reviewing BHaHAHA
-Steve is reviewing Gallery- think thats done
-Lucas and Beyhan reviewing knudaX -Lucas has started review
-Cottonmouth - Zac and Deborah will review, Zac has done a preliminary
AI overview
-Roland will review KapperX
-Boost - Steve is reviewer
-Two Punctures X Zac is reviewer
-Round of testing can now begin in earnest,
we have the complete set of thorns in place
-Steve is trying to run on Frank(enstein) to test on many types of GPUs
-Apptainer no longer needs root.

Announcements:
ET workshop is June 15-18 -Travel grants will not work via reimbursement

Atlassian is retiring Bitbucket Cloud Issues- Everything will shut
down by August
-Jira or github?
-Zach is going to test moving some into a dummy github page
-Should try and keep issue name the same
-If not done by August 1st we?ll switch to a Jira migration

Kyoto initiative
-A few hydro codes will be represented
-Jay and Zac are already involved for AsterX and IllinoisGRMHD (CPU)
-Beyhan voiced interest in representing GRHydro alongside these two
-Steve would also like to contribute
-We will revisit this around once a month to keep up to date

Unanswered questions on the mailing list - None

Tickets:
2929:  Loss of convergence with subcycling in time
2909: Remove TAGS from interface.ccl
2923: Adios2 compilation
-Steve replied has not heard back. Should already be fixed
1775:
-In the works, but we?re limited to versions
-Its rather large being 200 MB of source code
2921:
-No updates from Lucas, waiting for review
2922:
-Seems to be a bug in cuda 12.9
-Only a very hacky solution works

From nikolaiwallin at gmail.com  Thu Apr  2 11:09:04 2026
From: nikolaiwallin at gmail.com (Nikolai Wallin)
Date: Thu, 2 Apr 2026 09:09:04 -0700
Subject: [Users] Meeting minutes for 2026-03-26
Message-ID: <CAHKMMx+Yg3hX+ygbad=OiGXVGW6_m6dpDVR=iDwVvU5N-Gaa=w@mail.gmail.com>

Release:
-Cottonmouth in master, Doc directory is non existent.
This needs to be fixed. Currently the fastest according to Steve.
-Two Punctures X should be added. Currently added to master
-Probably need to update the CI system to ensure numpy is installed.
Currently failing
-Overall release is making progress,
we need to get the reviews out of the way.
-Need to add some work to the website front page.
-Peter is reviewing BHaHAHA
-Steve is reviewing Gallery- think thats done
-Lucas and Beyhan reviewing knudaX -Lucas has started review
-Cottonmouth - Zac and Deborah will review, Zac has done a preliminary
AI overview
-Roland will review KapperX
-Boost - Steve is reviewer
-Two Punctures X Zac is reviewer
-Round of testing can now begin in earnest,
we have the complete set of thorns in place
-Steve is trying to run on Frank(enstein) to test on many types of GPUs
-Apptainer no longer needs root.

Announcements:
ET workshop is June 15-18 -Travel grants will not work via reimbursement

Atlassian is retiring Bitbucket Cloud Issues- Everything will shut
down by August
-Jira or github?
-Zach is going to test moving some into a dummy github page
-Should try and keep issue name the same
-If not done by August 1st we?ll switch to a Jira migration

Kyoto initiative
-A few hydro codes will be represented
-Jay and Zac are already involved for AsterX and IllinoisGRMHD (CPU)
-Beyhan voiced interest in representing GRHydro alongside these two
-Steve would also like to contribute
-We will revisit this around once a month to keep up to date

Unanswered questions on the mailing list - None

Tickets:
2929:  Loss of convergence with subcycling in time
2909: Remove TAGS from interface.ccl
2923: Adios2 compilation
-Steve replied has not heard back. Should already be fixed
1775:
-In the works, but we?re limited to versions
-Its rather large being 200 MB of source code
2921:
-No updates from Lucas, waiting for review
2922:
-Seems to be a bug in cuda 12.9
-Only a very hacky solution works

From marcobrito at ua.pt  Thu Apr  2 11:10:36 2026
From: marcobrito at ua.pt (Marco Brito)
Date: Thu, 2 Apr 2026 16:10:36 +0000
Subject: [Users] Bug in QuasiLocalMeasures
Message-ID: <f23df2971fd648df834580f636e451f5@ua.pt>

Good afternoon,

I found a bug in the QuasiLocalMeasures thorn, in the computation of the ADM momentum. In the source code file qlm_analyse.F90, in line 361 there is a plus sign instead of a minus sign, contradicting the ADM momentum formula: P^i_{ADM} = \frac{1}{8\pi}\lim_{r\to\infty} \oint_S (K^i_j - \delta^i_j K)dS^j [1]

As expected for slices where K \neq 0  one was getting a wrong result. The correction is easy, just replace the plus sign by a minus sign. In any case you can find a patch file in attachment. Should I in any case file a ticket or a Pull Request somewhere regarding this?


Best regards,

Marco Brito.

[1] M. Alcubierre, Introduction to 3+1 Numerical Relativity, International Series of Monographs on Physics (Oxford University Press, Oxford, 2008).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20260402/566674a6/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: qlm_patch
Type: application/octet-stream
Size: 512 bytes
Desc: qlm_patch
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20260402/566674a6/attachment.obj>

From users at einsteintoolkit.org  Mon Apr  6 15:18:01 2026
From: users at einsteintoolkit.org (users at einsteintoolkit.org)
Date: Mon, 06 Apr 2026 15:18:01 -0500
Subject: [Users] Agenda for Thursday's Meeting
Message-ID: <69d414f9.HBkFgffv7OwyUBNo%users@einsteintoolkit.org>

Please update the Wiki with agenda items for Thursday's meeting. Thanks!

https://docs.einsteintoolkit.org/et-docs/meeting_agenda

--The Maintainers

From users at einsteintoolkit.org  Wed Apr  8 17:15:01 2026
From: users at einsteintoolkit.org (users at einsteintoolkit.org)
Date: Wed, 08 Apr 2026 17:15:01 -0500
Subject: [Users] Einstein Toolkit Meeting Reminder
Message-ID: <69d6d365.rQ+JugxTkOrhYsoD%users@einsteintoolkit.org>

Hello,

Please consider joining the weekly Einstein Toolkit phone call at
9:00 am US central time on Thursdays. For details on how to connect
and what agenda items are to be discussed, use the link below.

https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call

--The Maintainers

From lucas.t.s.carneiro at gmail.com  Thu Apr  9 10:39:25 2026
From: lucas.t.s.carneiro at gmail.com (Lucas Timotheo Sanches)
Date: Thu, 9 Apr 2026 10:39:25 -0500
Subject: [Users] Meeting Minutes for 2026-04-09
Message-ID: <CAFOyWAh_9EQHjQc1fe+ayHO5vXHFO93qOQvD0PhWFMmAEAZY=A@mail.gmail.com>

Attendees: Peter Diener, Lucas T. Sanches (minutes), Steven R Brandt (chair),
           Roland, Jordan Nicoules, Cheng Hsin Cheng, Maxwell Rizzo,
           Nikolai Wallin.

# Pre agenda topics

Because Jordan joined, we started discussing issue 2927. He found a bug with
GCC versions. Steve thinks that this is due to a bug when CST gets generated
into code. He says he should be able to find the problem.

We also discussed issue 2912, due to the Llama inconsistent volume forms.
Jordan says he has a fix for the issue though, so he promised to send a PR to
fix it soon.

# ET_2026_05 release

## Gallery runs

- The BH-NS gallery example is done now.
- No other gallery runs have started.

## Test suite runners

- Anvil and Sunrise are giving problems on multiple thorns:
1. CCE export. Peter looked at the failures. Two HDF5 are generated but the
   test suite obviously cannot compare binaries, so it relies on a python
   script. It's likely that Python is the root issue here.
2. CanudaX NP Scalars
3. CottonmouthBSSNOK.

TwoPuncturesX is failing everywhere. Steve says the tests should be updated.
Some tests require matter, which we don't have for CarpetX on the release.

## Thorn Reviews

- BHaHAHA: Peter wants to create a par file with it included so he can
 understand all the options. Zach says he would like to chat about it
 and Peter wants to include it in SPHINCS BSSN.

- Boost: Peter tried to download and put it in his thornlist, but it is failing.
  Roland says we should actually decide what we are going to provide from boost.
  So far, it seems we need System, file system, math, parameter parsing, and
  FUKA also needs python, but that will be a headache.

- Cottonmouth: Lucas presented some prelim. results of a new Z4c recipe.
  Steve stated our intention to have this thorn proposed for inclusion.
  Steve also presented timing results.

# ET Workshop at UIUC

- Registration is open.
- Speakers have been invited. Many have responded.

# Atlassian ticket migration

- No updates as of yet, we still have time.

# Kyoto initiative

- IllinoisGRMHD had volunteers on their behalf. Zach is going to meet with
  them to discuss details. They may join next ETK call

# Mailing list questions

- Bug in QuasiLocalMeasures: Jordan will let his collaborator make a ticket.
- Range error setting parameter, on cluster: Jordan discussed this during the
  start of the call. A ticket will be made about it

# Tickets

- 2928: Steve asked the community's opinion on adding a gpu option to
  simfactory. Lucas and Cheng-Hsin would like that.
- 2923: Maybe upgrading ADIOS2 fixes the issue, but Cheng-Hsin had not
time to test it yet.
- 2910: Apply.
- 2921: Cheng-Hsin has further comments, he will add it.

From lucas.t.s.carneiro at gmail.com  Thu Apr  9 11:11:41 2026
From: lucas.t.s.carneiro at gmail.com (Lucas Timotheo Sanches)
Date: Thu, 9 Apr 2026 11:11:41 -0500
Subject: [Users] Meeting Minutes for 2026-04-09: Amended
Message-ID: <CAFOyWAguM3r05AiGTugLQtey8VLT4T6UmDeUHfQBsh4RmZ9jWg@mail.gmail.com>

Attendees: Peter Diener, Lucas T. Sanches (minutes), Steven R Brandt (chair),
           Roland, Jordan Nicoules, Cheng Hsin Cheng, Maxwell Rizzo,
           Nikolai Wallin.

# Pre agenda topics

Because Jordan joined, we started discussing issue 2927. He found a bug with
GCC versions. Steve thinks that this is due to a bug when CST gets generated
into code. He says he should be able to find the problem.

We also discussed issue 2912, due to the Llama inconsistent volume forms.
Jordan says he has a fix for the issue though, so he promised to send a PR to
fix it soon.

# ET_2026_05 release

## Gallery runs

- The BH-NS gallery example is done now.
- No other gallery runs have started.

## Test suite runners

- Anvil and Sunrise are giving problems on multiple thorns:
1. CCE export. Peter looked at the failures. Two HDF5 are generated but the
   test suite obviously cannot compare binaries, so it relies on a python
   script. It's likely that Python is the root issue here.
2. CanudaX NP Scalars
3. CottonmouthBSSNOK.

TwoPuncturesX is failing everywhere. Steve says the tests should be updated.
Some tests require matter, which we don't have for CarpetX on the release.

## Thorn Reviews

- BHaHAHA: Peter wants to create a par file with it included so he can
 understand all the options. Zach says he would like to chat about it
 and Peter wants to include it in SPHINCS BSSN.

- Boost: Peter tried to download and put it in his thornlist, but it is failing.
  Roland says we should actually decide what we are going to provide from boost.
  So far, it seems we need System, file system, math, parameter parsing.
  Sam Tootle clarified that Fuka only requires Boost Python for the
Python libraries
  which are built manually by the user and are not required by the importer.

- Cottonmouth: Lucas presented some prelim. results of a new Z4c recipe.
  Steve stated our intention to have this thorn proposed for inclusion.
  Steve also presented timing results.

# ET Workshop at UIUC

- Registration is open.
- Speakers have been invited. Many have responded.

# Atlassian ticket migration

- No updates as of yet, we still have time.

# Kyoto initiative

- IllinoisGRMHD had volunteers on their behalf. Zach is going to meet with
  them to discuss details. They may join next ETK call

# Mailing list questions

- Bug in QuasiLocalMeasures: Jordan will let his collaborator make a ticket.
- Range error setting parameter, on cluster: Jordan discussed this during the
  start of the call. A ticket will be made about it

# Tickets

- 2928: Steve asked the community's opinion on adding a gpu option to
  simfactory. Lucas and Cheng-Hsin would like that.
- 2923: Maybe upgrading ADIOS2 fixes the issue, but Cheng-Hsin had not
time to test it yet.
- 2910: Apply.
- 2921: Cheng-Hsin has further comments, he will add it.

From users at einsteintoolkit.org  Mon Apr 13 15:18:01 2026
From: users at einsteintoolkit.org (users at einsteintoolkit.org)
Date: Mon, 13 Apr 2026 15:18:01 -0500
Subject: [Users] Agenda for Thursday's Meeting
Message-ID: <69dd4f79.Zw8Y1/kmLMS4sUql%users@einsteintoolkit.org>

Please update the Wiki with agenda items for Thursday's meeting. Thanks!

https://docs.einsteintoolkit.org/et-docs/meeting_agenda

--The Maintainers