Trac October 2022

trac@lists.einsteintoolkit.org

1 participants
87 discussions

#2653: Carpet PreSync triggering syncs for all refinement levels unnecessarily
by Samuel Cupp 05 Oct '22

05 Oct '22

#2653: Carpet PreSync triggering syncs for all refinement levels unnecessarily Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: major Component: Carpet In Carpet, the function `PreSyncGroups` calls `SyncProlongateGroups` to automatically sync variables based on the reads and writes. I include the function below. ``` void PreSyncGroups(cFunctionData *attribute, cGH *cctkGH, const std::vector<int> &pre_groups) { DECLARE_CCTK_PARAMETERS; assert(not CCTK_EQUALS(presync_mode, "off") and not CCTK_EQUALS(presync_mode, "warn-only")); if (pre_groups.empty()) return; if (reflevel > 0) { // recurse to check that all coarsers levels are properly SYNCed CCTK_REAL previous_time = cctkGH->cctk_time; const int parent_reflevel = reflevel - 1; BEGIN_GLOBAL_MODE(cctkGH) { ENTER_LEVEL_MODE(cctkGH, parent_reflevel) { cctkGH->cctk_time = tt->get_time(mglevel, reflevel, timelevel); PreSyncGroups(attribute, cctkGH, pre_groups); } LEAVE_LEVEL_MODE; } END_GLOBAL_MODE; cctkGH->cctk_time = previous_time; } // ask Carpet to do the SYNC, this will apply BC as well SyncProlongateGroups(cctkGH, pre_groups, attribute); } ``` Starting at line 11, the code claims to “check that all coarser levels are properly SYNCed”. However, this “check” amounts to blindly syncing every coarser refinement level! This is very inefficient and a huge performance problem that grows with the number of refinement levels. With a parfile for a binary black hole simulation with BaikalVacuum using 12 refinement levels, I found that each iteration was taking 6 times longer. For now in my work, I am commenting lines 11-24 out and assuming that coarser levels are fine. However, this definitely needs to be fixed in a more ‘proper’ way. This ties into a similar issue I ran into when I didn’t have boundary conditions registered \(due to my own mistake\). Since boundaries weren’t getting filled, every READ\(everywhere\) would trigger sync\+BCs, but there were no BCs to apply. Therefore, even if it _should_ have been synced already, it would sync _again_ with the next READ\(everywhere\) because there were no BCs applied. As an aside, I feel that this case might have needed to generate an error somewhere, as a READ\(everywhere\) was running fine even though it was only valid for interior \+ ghosts. I propose that a reasonable fix would be to add a check in SyncProlongateGroups to see what is actually needed. This function eventually ends up with a list of `goodgroups` that are to be synced and have boundary conditions applied. Once this preliminary list is constructed, PreSync should further refine this list into those that need syncs and those that need BCs using validities. Then, it can pass two different lists to each, if necessary. I don’t know if there’s another way for PreSync simulations to get to SyncGroups other than through this function, so there may have to be a check in SyncGroups if that is the case. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2653/carpet-presync-tr…

1 0

#2651: New warning message for missing boundary conditions
by Samuel Cupp 04 Oct '22

04 Oct '22

#2651: New warning message for missing boundary conditions Reporter: Samuel Cupp Status: resolved Milestone: Version: Type: task Priority: minor Component: Carpet Changes (by Samuel Cupp): status: resolved (was new) Comment (by Samuel Cupp): The new error message has been approved and merged into master. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2651/new-warning-messa…

1 0

#2649: Missing function Cartoon2D interface.ccl
by Samuel Cupp 04 Oct '22

04 Oct '22

#2649: Missing function Cartoon2D interface.ccl Reporter: Samuel Cupp Status: resolved Milestone: Version: Type: bug Priority: minor Component: Cactus Changes (by Samuel Cupp): status: resolved (was new) Comment (by Samuel Cupp): Pull request resolving issue has been merged into master. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2649/missing-function-…

1 0

#2649: Missing function Cartoon2D interface.ccl
by Samuel Cupp 04 Oct '22

04 Oct '22

#2649: Missing function Cartoon2D interface.ccl Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: minor Component: Cactus Comment (by Samuel Cupp): Pull request has been merged in. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2649/missing-function-…

1 0

#2649: Missing function Cartoon2D interface.ccl
by Roland Haas 03 Oct '22

03 Oct '22

#2649: Missing function Cartoon2D interface.ccl Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: minor Component: Cactus Comment (by Roland Haas): Please apply. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2649/missing-function-…

1 0

#2649: Missing function Cartoon2D interface.ccl
by Gabriele Bozzola 03 Oct '22

03 Oct '22

#2649: Missing function Cartoon2D interface.ccl Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: minor Component: Cactus Comment (by Gabriele Bozzola): I just ran into this problem and found that the latest version of the intel compiler will error out by default. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2649/missing-function-…

1 0

#2639: CarpetLib internal error with PreSync and analysis thorns
by Samuel Cupp 03 Oct '22

03 Oct '22

#2639: CarpetLib internal error with PreSync and analysis thorns Reporter: Samuel Cupp Status: new Milestone: Version: development version Type: bug Priority: major Component: Carpet Comment (by Samuel Cupp): Following up on this ticket, I know you made a [branch](https://bitbucket.org/eschnett/carpet/src/97e6f6a5544373d4c1f05f617… with a partial fix, Roland. Do you think that’s ok to push in, or do we need to do something more? Since the behavior is broken now, I don’t see why we shouldn’t at least merge in a partial fix. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2639/carpetlib-interna…

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Trac October 2022