Changes (by Miren Radia):

This issue is related to #2890 but I’ve now tried running almost the same parameter file again but with the main versions of CarpetX/SpacetimeX rather than Liwei’s development branches.

I am trying to run a BH binary simulation with CarpetX on tursa (Nvidia A100 GPUs) but I seem to be running into issues when the PunctureTracker thorn is enabled.

Please find attached the following:

• My thornlist: spacetimex.th

• My parameter file: test-bbh.par (small grid configuration to make it easier to reproduce the issue). This is more or less the same as the one in #2890 but with the following changes:

  • CarpetX::poison_undefined_values = no I had to disable this as when I had it set to yes, I got the following error (although it looks likely related):

    ERROR from host tu-c0r0n87 process 0 in thorn CarpetX, file /mnt/lustre/tursafs1/home/dp415/dp415/dc-radi1/ETK/Cactus-CarpetX-main/arrangements/CarpetX/CarpetX/src/valid.cxx:552: -> CallFunction iteration 0 PunctureTracker_SetupGroup: PunctureTracker::PunctureTracker_Setup checking output: Grid array "BOXINBOX::position_x[2]" has 1 nans on time level 0; expected valid The interior is valid because: CallFunction iteration 0 PunctureTracker_SetupGroup: PunctureTracker::PunctureTracker_Setup: Mark output variables as valid. The outer boundary is valid because: CallFunction iteration 0 PunctureTracker_SetupGroup: PunctureTracker::PunctureTr

    --
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2892/spacetimex-gpu-error-in-puncturetracker