Dear all,
A couple of weeks ago, I found out that there was a bug in CarpetRegrid2. Namely, it does not properly check for a change of radius of the refinement levels (when they are steered by a Trigger for instance). Looking a bit closer, it seems that the computed `drfac` in `CarpetRegrid2/src/regrid.cc` is nan, and thus `do_recompose` is not set to true. In particular, it means it is possible to change the radii if something else triggers a regridding (for instance, changing the number of active levels).
However, while I was trying to understand and fix this bug, it appeared there might be a deeper issue. I'm struggling to understand if it may be related to memory, compilation, ...
It would be helpful if any of you had some insight to share, or could try to replicate the issue. To that effect, I'm attaching a minimal parfile adapted from CarpetRegrid2/test/regrid2_granularity.par, as well as a modified regrid.cc file that should allow you to reproduce the situation. More details on the situation below.
Best,
Jordan
----------------------------
----- More details -----
----------------------------
To figure out why drfac is nan, I added manual debugging information prints in the code. Depending on what information I ask, and in what order, I don't get consistent nor sensible results. I'm reaching out here first instead of opening a ticket because I can't pinpoint the precise issue.
I haven't tried different compilers (I have gcc 11.4.0 locally), but this is happening even with a debug build (see attached make.config.defn file).
More precisely, printing `rad`, `oldrad`, `rad - oldrad` is fine (done through std::cout).
Printing (CCTK_VINFO) the numerator and denominator of `drfac` separately gives the right values for them, but the quotient `drfac` is wrong (it is 0, and not nan anymore, even when it shouldn't).
If I create new local variables
```
CCTK_REAL const num = sqrt(sum(ipow(rad - oldrad, 2))); CCTK_REAL const den = sqrt(sum(ipow(oldrad, 2))); ``` and print them, they both contain the same value. It looks like the values are as if it uses rad=0 and whatever oldrad used to have before (which can be thus changed, depending on the order of the print lines). Finally, if I do print a lot of other info (in particular the intermediate computation steps), it seems it finally gets drfac right.
I had a look at CarpetLib's vect.hh and defs.hh too, but I can't seem to understand what is going on. Note that there might be another small bug in the thorn, but fixing that didn't impact what I'm describing. I'm just attaching the other modified files (schedule.ccl and initialise.cc) for completeness, as it may lead to a fix eventually.