I have been investigating the source of the slowdown for the evolution of a BBH using BaikalVacuum when PreSync is turned on. The largest problem was reported in Ticket #2653, but the code is still noticeably slower than without PreSync. I have tracked the extra sync calls to restriction. There is one group which is only written, never read. In the old way, these syncs were manually scheduled in the schedule.ccl. PreSync (properly) syncs this group at restriction at every refinement level. The observed behavior shows it

1. Syncs the group aux_variables from the coarsest to finest level

Then, it begins heading back up the levels, presumably to begin the restriction operation. However, it then

2. syncs the groups aux_variables and evol_variables from finest to coarsest, skipping the finest level

The syncs in 2 are not present in the old way. Either the old way wasn’t properly syncing during restriction, or PreSync is introducing extra syncs into this operation. I’m inclined to believe the latter. My guess as to the possible source of this is the code

// Restrict a refinement level
void ggf::ref_restrict_all(comm_state &state, int const tl, int const rl,
                           int const ml--

Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2656/presync-syncing-more-than-old-code