[ET Trac] [Einstein Toolkit] #1256: scheduling MoL_PostStep in Post_Recover_Variables

[Einstein Toolkit]

#1256: scheduling MoL_PostStep in Post_Recover_Variables
---------------------+------------------------------------------------------
  Reporter:  rhaas   |       Owner:     
      Type:  task    |      Status:  new
  Priority:  major   |   Milestone:     
 Component:  Cactus  |     Version:     
Resolution:          |    Keywords:  MoL
---------------------+------------------------------------------------------

Comment (by rhaas):

 Con2Prim is an oddball routine that does not really behave nicely
 unfortunately. Depending on the EOS used it might even modify the
 conservatives.

 With the MoL scheduling as at is currently, I don't think that Con2Prim
 can run in PostStepModify. The reason is that Con2Prim needs to run in
 postregrid where currently poststepmodify does not seem to run.
 Essentially con2prim needs to run whenever the conservative varialbles
 (ie. dens, scon, tau, namely GRHydro's evolved variables change). In that
 sense it is similar to say the adm constraints. The problem is that
 calling con2prim multiple times changes the result.

 For not calling mol_poststep in post_recover_variables the argument is
 that we want to restore the state of the simulation bit identical to what
 it was at checkpoint time. So if we recompute something (admbase
 variables, constraints, primitives) we must be sure that we will get
 precisely the same values as we got at checkpoint time.

 Also since we already know that we cannot call MoL_PostStep on the past
 timelevels (since SYNC does not work), calling it on the current timelevel
 only is strange. Any constraints that might need to be applied should
 already have been applied at the time the checkpoint was written.

 To my manner of thinking, checkpoint/restore should not call any
 additional functions but just restore the simulation to precisely the
 state it was in when it was checkpointed, so if the constraints/quantities
 that MoL_PostStep enforces were wrong at checkpointing time, then they
 should still be wrong at restore time (since there are routines that run
 between mol_poststep and the point where the simulation is checkpointed
 and that might modify the variables). Variables should be either
 checkpointed or if one wants to be clever and recompute them after
 checkpointing then the routine to recompute should be scheduled explicitly
 in POST_RECOVER_VARIABLES (based on the argument that the thorn apparently
 wants to know about checkpointing so can be expected to take special
 measures).

 Is there a case where we *require* MoL_PostStep because it is the only way
 to fill in data (when recovering on a different number processors?)? Or
 was it put there just to be sure? Note that it was not always scheduled in
 POST_RECOVER_VARIABLES, this was done in
 changeset:137/CactusNumerical/MoL.
 {{{
 Schedule MoL_PostStep also at post_recover_variables.  This ensures
 that boundary conditions, ADM variables etc. are correct after
 recovering.
 }}}
 and that the comment in the schedule file says
 {{{
 ##############################################################
 ###  Schedule the PostStep parts after recovery            ###
 ###  so that symmetries are automatically done correctly.  ###
 ###  We may want to change this later as it could be       ###
 ###  expensive, but it's simplest for the moment.          ###
 ##############################################################
 }}}
 Currently we checkpoint the ADMBase variables.

 Currently the recovery routine reads in everything incl. ghost points (and
 ghost points are checkpointed). Currently it is possible to have
 space_mask and grhydro::atmosphere_mask differ in ghost zones and interior
 points (since GRHydro uses it to keep track of some information of values
 in eg densrhs which is only valid in the interior anyway and so never
 bothers to synchronize it). Adding the SYNC makes it possible to get bit
 identical recovery from checkpoints (if one leaves out mol_poststep).

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1256#comment:2>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit