[Users] Question about schedule options

[Roland Haas]

Hello Jordan,

Scheduling in Carpet with subcycling in time involved is quite tricky.

Let me try and give rundown.

* first to realize is that the schedule that you see printed out to
  screen is somewhat of a fiction

* second to realize is that Carpet will execute the schedule for each
  bin (eg ANALYSIS) level-by-level. It will not however always execute
  all levels it can in one bin before executing the next bin (namely
  that only happens beween EVOL and POSTEP and at some points between
  INITIAL and POSTPOSTINITIAL [and EVOL]). Details in
  `Carpet/Carpet/src/Evolve.cc` or
  https://github.com/EinsteinToolkit/Carpet/blob/master/doc/scheduling.pdf
  pg 8

* a GLOBAL-EARLY routine will be executed along with the regular
  routines when the FIRST refinement level is processed (whichever one
  that may be)

* a GLOBAL-LATE routine will be executed along with the regular
  routines when the LAST refinement is processed (whichever one that
  may be)

* BEFORE / AFTER statements are only effective within a single schedule
  pass. So a routine A that is GLOBAL-EARLY and is AFTER a LOCAL
  routine B *will* execute after routine B in that FIRST refinement
  level. But *will* have executed BEFORE any instance of routine B on
  later level (it won't execute more than once though)

* the LOOP-LOCAL option (which can only be used with GLOBAL, SINGLEMAP
  or LEVEL routines) means that Carpet will insert an extra loop over
  all the refinement levels and will call the scheduled routine on *all*
  levels, all the time. Even if a LOCAL routine at that iteration would
  not be. So it is somewhat more expensive than a LOCAL routine.

If you want to compute something for Trigger to use, I would suggest to
compute it in POSTSTEP rather than ANALYSIS (ANALYSIS is not supposed
to compute anything that is used by others, really, only analyze data
for output). Note that you will have to compute the quantity all the
time, not just in iterations that say you want to use it (b/c the
output may use data on refinement levels that would have been computed
and that you skipped).

Yours,
Roland

> [CAUTION: Non-UBC Email]
> 
> Hi all,
> 
> 
> I have a question about scheduling routines in the analysis bin, more
> particularly about the options you can provide. Is there any downside
> in scheduling a function as "global-early loop-local", compared to
> the default "local"? Like a loss of performance, for instance if
> there end up being multiple such functions without non-trivial order?
> 
> 
> My issue is, Trigger::Trigger_Check is scheduled as global-early, but
> I would like to compute GFs before (and reductions, but that is not
> the problematic part). The "before" directive seems to only properly
> apply if I make my routine "global-early loop-local", which I can
> understand. If there's no additional overhead, I don't mind changing
> the options of my routine. If it may turn out to have impact on
> performance, I may prefer to introduce conditional statements or find
> other workarounds.
> 
> 
> Best,
> 
> 
> Jordan


Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .