- by "naive", we mean essentially the following
in schedule.ccl we have
STORAGE: shift_t0[1]
in interface.ccl we have
CCTK_REAL shift_t0 type=gf timelevels=1 tags='tensortypealias="U" tags='prolongation="none"' { betax_t0 betay_t0 betaz_t0 } "shift vector at t=0"
and we set its value (on the whole grid) at a function call at CCTK_INITIAL after ADMBase_PostInitial
Does this happen only after regridding? By not prolongating the refinement boundaries, it's not clear how the "new" bits of the grid are supposed to be filled when the fine grids move. Could this be the problem? If you give the gridfunction 3 timelevels and remove "prolongation=none", do the NaNs go away?
it seems to happen only after regridding, yes. we did try precisely that (using 3 timelevels and prolongating) but the NaNs still appeared...
The past timelevels need to be initialised. By default, Carpet will poison them with NaNs so that you can see when undefined values are used. Theoretically, if there is no time evolution of this variable, you should be able to use a single timelevel and use a tag 'prolongation="copy"', which should perform only spatial prolongation. Let me know what happens if you try this. I expect you will get an assertion failure from Carpet,
indeed, this is what happens. i got:
WARNING level 0 from host meurglysIII process 0 while executing schedule bin (none), routine (no thorn)::(no routine) in thorn CarpetLib, file /home/mzilhao/dev/ET/Cactus/arrangements/Carpet/CarpetLib/src/data.cc:615: -> There is no vertex-centred stencil for op="LAGRANGE" or op="COPY" with order_space not in {1, 3, 5, 7, 9, 11} -------------------------------------------------------------------------- mpirun noticed that process rank 1 with PID 11602 on node meurglysIII exited on signal 6 (Aborted). --------------------------------------------------------------------------
and then you might want to try cherry-picking this commit:
cd repos/Carpet git cherry-pick d13a6e245a3d7d2b575094fd0def540d510a166c
which disables some code in Carpet that disables spatial prolongation in the case of "op_copy". The problem here likely resulted in confusion due to the operator being badly named (https://martinfowler.com/bliki/TwoHardThings.html). In arrangements/Carpet/CarpetLib/src/operators.hh, the implication is that the "copy" in op_copy refers to the time interpolation only, but the code in patched in the above commit assumes that "copy" means don't interpolate at all. It's not clear how this can make sense when the data comes from another refinement level, which will always have a different resolution, but maybe there is some case where this would have made sense.
Erik, should this commit be merged to master?
with this commit it seems to be working fine, yes! unless there is some other way of doing this, it would be great if this commit could be merged to master.
many thanks, Miguel