Hello Jordan,
Scheduling in Carpet with subcycling in time involved is quite tricky.
Let me try and give rundown.
* first to realize is that the schedule that you see printed out to screen is somewhat of a fiction
* second to realize is that Carpet will execute the schedule for each bin (eg ANALYSIS) level-by-level. It will not however always execute all levels it can in one bin before executing the next bin (namely that only happens beween EVOL and POSTEP and at some points between INITIAL and POSTPOSTINITIAL [and EVOL]). Details in `Carpet/Carpet/src/Evolve.cc` or https://github.com/EinsteinToolkit/Carpet/blob/master/doc/scheduling.pdf pg 8
* a GLOBAL-EARLY routine will be executed along with the regular routines when the FIRST refinement level is processed (whichever one that may be)
* a GLOBAL-LATE routine will be executed along with the regular routines when the LAST refinement is processed (whichever one that may be)
* BEFORE / AFTER statements are only effective within a single schedule pass. So a routine A that is GLOBAL-EARLY and is AFTER a LOCAL routine B *will* execute after routine B in that FIRST refinement level. But *will* have executed BEFORE any instance of routine B on later level (it won't execute more than once though)
* the LOOP-LOCAL option (which can only be used with GLOBAL, SINGLEMAP or LEVEL routines) means that Carpet will insert an extra loop over all the refinement levels and will call the scheduled routine on *all* levels, all the time. Even if a LOCAL routine at that iteration would not be. So it is somewhat more expensive than a LOCAL routine.
If you want to compute something for Trigger to use, I would suggest to compute it in POSTSTEP rather than ANALYSIS (ANALYSIS is not supposed to compute anything that is used by others, really, only analyze data for output). Note that you will have to compute the quantity all the time, not just in iterations that say you want to use it (b/c the output may use data on refinement levels that would have been computed and that you skipped).
Yours, Roland
[CAUTION: Non-UBC Email]
Hi all,
I have a question about scheduling routines in the analysis bin, more particularly about the options you can provide. Is there any downside in scheduling a function as "global-early loop-local", compared to the default "local"? Like a loss of performance, for instance if there end up being multiple such functions without non-trivial order?
My issue is, Trigger::Trigger_Check is scheduled as global-early, but I would like to compute GFs before (and reductions, but that is not the problematic part). The "before" directive seems to only properly apply if I make my routine "global-early loop-local", which I can understand. If there's no additional overhead, I don't mind changing the options of my routine. If it may turn out to have impact on performance, I may prefer to introduce conditional statements or find other workarounds.
Best,
Jordan
Yours, Roland