[Users] Fwd: PreSync and recovery from checkpoint

[Roland Haas]

Hello Gabriele,

I seem to have trouble sending email to the ET mailing list server
using my regular UIUC email address, so maybe using GMail will work
better.

Yours,
Roland

Begin forwarded message:

Date: Sat, 1 Jan 2022 10:36:00 -0500
From: Roland Haas <rhaas at illinois.edu>
To: Gabriele Bozzola <bozzola.gabriele at gmail.com>
Cc: Einstein Toolkit Users <users at einsteintoolkit.org>
Subject: Re: [Users] PreSync and recovery from checkpoint


Hello Gabriele,

> I have a fork of ProcaEvolve (
> https://urldefense.com/v3/__https://bitbucket.org/Sbozzolo/proca/src/master/__;!!DZ3fjg!oMcQru84TKsWMoY2YFsLTkLW7WfZuSfD9TnxLZdhbUwY8BR_fjy2ZykIwaXVaVGa$ )
> where I added the READS/WRITES statements for
> PreSync and other features. One of such features
> is computing the constraint, but I found out that there
> is some problem when PreSync is running with
> `mixed-error`.  
hmm.

> I have a write statement for chi in CCTK_INITIAL, which
> is not called when restarting a simulation. Nonetheless,
> chi should be available everywhere because it is read
> from a checkpoint. What is the correct thing to do in
> this scenario?  
When grid functions are read in from a checkpoint then they are marked
as being valid everywhere. This is done in CarpetIOHDF5/src/Input.cc in
the Recover function in the block that starts with 

// check that all variables have been read completely on this
mglevel/reflevel

specifically in line 793 of that file:

gf->set_valid(mglevel, reflevel, tl, CCTK_VALID_EVERYWHERE);

If these variables are not checkpointed (likely, given that they are
only constraints and thus not usually considered "important" to
checkpoint) then you would have to add code to say
POST_RECOVER_VARIABLES to initialize them (eg to 0). This is a good
idea to do even without considering PreSync since it avoids having
random junk data (or poison) in the constraint output for a couple of
steps (depending on mesh refinement details) after recovery.

Something like:

schedule Proca_ClearConstraints AT POST_RECOVER_VARIABLES
{
  LANG: Fortran
} "Initialize constraint values after recovery"

You can also try to to Proca_Constraints in post-recover variables but,
since the constraints require derivatives and those cannot be
(correctly) computed in post-recover-variables, the constraints for the
checkpointed timestep will (possibly I would have to think about
exactly how checkpointing, prolongation and time stepping
interact) differ before and after recovery. 

Also POST_RECOVER_VARIABLES only runs the scheduled function on
timelevel 0 (ie current time) and not any past timelevels so if there
are multiple time levels to the constraint variables, only tl=0 will be
initialized.

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.


-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20220103/f8edbe11/attachment.bin