#1256: scheduling MoL_PostStep in Post_Recover_Variables ---------------------+------------------------------------------------------ Reporter: rhaas | Owner: Type: task | Status: new Priority: major | Milestone: Component: Cactus | Version: Resolution: | Keywords: MoL ---------------------+------------------------------------------------------
Comment (by rhaas):
Con2Prim is an oddball routine that does not really behave nicely unfortunately. Depending on the EOS used it might even modify the conservatives.
With the MoL scheduling as at is currently, I don't think that Con2Prim can run in PostStepModify. The reason is that Con2Prim needs to run in postregrid where currently poststepmodify does not seem to run. Essentially con2prim needs to run whenever the conservative varialbles (ie. dens, scon, tau, namely GRHydro's evolved variables change). In that sense it is similar to say the adm constraints. The problem is that calling con2prim multiple times changes the result.
For not calling mol_poststep in post_recover_variables the argument is that we want to restore the state of the simulation bit identical to what it was at checkpoint time. So if we recompute something (admbase variables, constraints, primitives) we must be sure that we will get precisely the same values as we got at checkpoint time.
Also since we already know that we cannot call MoL_PostStep on the past timelevels (since SYNC does not work), calling it on the current timelevel only is strange. Any constraints that might need to be applied should already have been applied at the time the checkpoint was written.
To my manner of thinking, checkpoint/restore should not call any additional functions but just restore the simulation to precisely the state it was in when it was checkpointed, so if the constraints/quantities that MoL_PostStep enforces were wrong at checkpointing time, then they should still be wrong at restore time (since there are routines that run between mol_poststep and the point where the simulation is checkpointed and that might modify the variables). Variables should be either checkpointed or if one wants to be clever and recompute them after checkpointing then the routine to recompute should be scheduled explicitly in POST_RECOVER_VARIABLES (based on the argument that the thorn apparently wants to know about checkpointing so can be expected to take special measures).
Is there a case where we *require* MoL_PostStep because it is the only way to fill in data (when recovering on a different number processors?)? Or was it put there just to be sure? Note that it was not always scheduled in POST_RECOVER_VARIABLES, this was done in changeset:137/CactusNumerical/MoL. {{{ Schedule MoL_PostStep also at post_recover_variables. This ensures that boundary conditions, ADM variables etc. are correct after recovering. }}} and that the comment in the schedule file says {{{ ############################################################## ### Schedule the PostStep parts after recovery ### ### so that symmetries are automatically done correctly. ### ### We may want to change this later as it could be ### ### expensive, but it's simplest for the moment. ### ############################################################## }}} Currently we checkpoint the ADMBase variables.
Currently the recovery routine reads in everything incl. ghost points (and ghost points are checkpointed). Currently it is possible to have space_mask and grhydro::atmosphere_mask differ in ghost zones and interior points (since GRHydro uses it to keep track of some information of values in eg densrhs which is only valid in the interior anyway and so never bothers to synchronize it). Adding the SYNC makes it possible to get bit identical recovery from checkpoints (if one leaves out mol_poststep).