Trac

trac@lists.einsteintoolkit.org

8708 discussions

#2653: Carpet PreSync triggering syncs for all refinement levels unnecessarily
by Samuel Cupp 05 Oct '22

05 Oct '22

#2653: Carpet PreSync triggering syncs for all refinement levels unnecessarily Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: major Component: Carpet Comment (by Samuel Cupp): I also feel I should add that I’m not convinced that this recursive behavior is necessary. Do we actually need to do this? -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2653/carpet-presync-tr…

1 0

#2653: Carpet PreSync triggering syncs for all refinement levels unnecessarily
by Samuel Cupp 05 Oct '22

05 Oct '22

#2653: Carpet PreSync triggering syncs for all refinement levels unnecessarily Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: major Component: Carpet In Carpet, the function `PreSyncGroups` calls `SyncProlongateGroups` to automatically sync variables based on the reads and writes. I include the function below. ``` void PreSyncGroups(cFunctionData *attribute, cGH *cctkGH, const std::vector<int> &pre_groups) { DECLARE_CCTK_PARAMETERS; assert(not CCTK_EQUALS(presync_mode, "off") and not CCTK_EQUALS(presync_mode, "warn-only")); if (pre_groups.empty()) return; if (reflevel > 0) { // recurse to check that all coarsers levels are properly SYNCed CCTK_REAL previous_time = cctkGH->cctk_time; const int parent_reflevel = reflevel - 1; BEGIN_GLOBAL_MODE(cctkGH) { ENTER_LEVEL_MODE(cctkGH, parent_reflevel) { cctkGH->cctk_time = tt->get_time(mglevel, reflevel, timelevel); PreSyncGroups(attribute, cctkGH, pre_groups); } LEAVE_LEVEL_MODE; } END_GLOBAL_MODE; cctkGH->cctk_time = previous_time; } // ask Carpet to do the SYNC, this will apply BC as well SyncProlongateGroups(cctkGH, pre_groups, attribute); } ``` Starting at line 11, the code claims to “check that all coarser levels are properly SYNCed”. However, this “check” amounts to blindly syncing every coarser refinement level! This is very inefficient and a huge performance problem that grows with the number of refinement levels. With a parfile for a binary black hole simulation with BaikalVacuum using 12 refinement levels, I found that each iteration was taking 6 times longer. For now in my work, I am commenting lines 11-24 out and assuming that coarser levels are fine. However, this definitely needs to be fixed in a more ‘proper’ way. This ties into a similar issue I ran into when I didn’t have boundary conditions registered \(due to my own mistake\). Since boundaries weren’t getting filled, every READ\(everywhere\) would trigger sync\+BCs, but there were no BCs to apply. Therefore, even if it _should_ have been synced already, it would sync _again_ with the next READ\(everywhere\) because there were no BCs applied. As an aside, I feel that this case might have needed to generate an error somewhere, as a READ\(everywhere\) was running fine even though it was only valid for interior \+ ghosts. I propose that a reasonable fix would be to add a check in SyncProlongateGroups to see what is actually needed. This function eventually ends up with a list of `goodgroups` that are to be synced and have boundary conditions applied. Once this preliminary list is constructed, PreSync should further refine this list into those that need syncs and those that need BCs using validities. Then, it can pass two different lists to each, if necessary. I don’t know if there’s another way for PreSync simulations to get to SyncGroups other than through this function, so there may have to be a check in SyncGroups if that is the case. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2653/carpet-presync-tr…

1 0

#2651: New warning message for missing boundary conditions
by Samuel Cupp 04 Oct '22

04 Oct '22

#2651: New warning message for missing boundary conditions Reporter: Samuel Cupp Status: resolved Milestone: Version: Type: task Priority: minor Component: Carpet Changes (by Samuel Cupp): status: resolved (was new) Comment (by Samuel Cupp): The new error message has been approved and merged into master. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2651/new-warning-messa…

1 0

#2649: Missing function Cartoon2D interface.ccl
by Samuel Cupp 04 Oct '22

04 Oct '22

#2649: Missing function Cartoon2D interface.ccl Reporter: Samuel Cupp Status: resolved Milestone: Version: Type: bug Priority: minor Component: Cactus Changes (by Samuel Cupp): status: resolved (was new) Comment (by Samuel Cupp): Pull request resolving issue has been merged into master. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2649/missing-function-…

1 0

#2649: Missing function Cartoon2D interface.ccl
by Samuel Cupp 04 Oct '22

04 Oct '22

#2649: Missing function Cartoon2D interface.ccl Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: minor Component: Cactus Comment (by Samuel Cupp): Pull request has been merged in. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2649/missing-function-…

1 0

#2649: Missing function Cartoon2D interface.ccl
by Roland Haas 03 Oct '22

03 Oct '22

#2649: Missing function Cartoon2D interface.ccl Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: minor Component: Cactus Comment (by Roland Haas): Please apply. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2649/missing-function-…

1 0

#2649: Missing function Cartoon2D interface.ccl
by Gabriele Bozzola 03 Oct '22

03 Oct '22

#2649: Missing function Cartoon2D interface.ccl Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: minor Component: Cactus Comment (by Gabriele Bozzola): I just ran into this problem and found that the latest version of the intel compiler will error out by default. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2649/missing-function-…

1 0

#2639: CarpetLib internal error with PreSync and analysis thorns
by Samuel Cupp 03 Oct '22

03 Oct '22

#2639: CarpetLib internal error with PreSync and analysis thorns Reporter: Samuel Cupp Status: new Milestone: Version: development version Type: bug Priority: major Component: Carpet Comment (by Samuel Cupp): Following up on this ticket, I know you made a [branch](https://bitbucket.org/eschnett/carpet/src/97e6f6a5544373d4c1f05f617… with a partial fix, Roland. Do you think that’s ok to push in, or do we need to do something more? Since the behavior is broken now, I don’t see why we shouldn’t at least merge in a partial fix. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2639/carpetlib-interna…

1 0

#2652: ./ET_executable -O outputs about 19,000 lines
by Zach Etienne 30 Sep '22

30 Sep '22

#2652: ./ET_executable -O outputs about 19,000 lines Reporter: Zach Etienne Status: new Milestone: Version: Type: bug Priority: major Component: ``` -O[v], --describe-all-parameters : describes all the parameters. v makes this verbose, i.e., it gives a verbose description of all parameters. ``` When running the Einstein Toolkit executable with `--help`, the above command-line option appears. The 19,123 lines of output is terrifying, but could be made less so. For one thing, each element of a parameter array is shown individually, and many arrays have 100 elements. I think the output for arrays should be combined into a single line. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2652/et_executable-o-o…

1 0

#2651: New warning message for missing boundary conditions
by Samuel Cupp 30 Sep '22

30 Sep '22

#2651: New warning message for missing boundary conditions Reporter: Samuel Cupp Status: new Milestone: Version: Type: task Priority: minor Component: Carpet Changes (by Samuel Cupp): I have made a pull request adding a warning inside ApplyPhysicalBCsForGroupI for when variables have no boundary conditions registered. Originally, there was a _cout_ statement only if a preprocessor flag was defined. I have found that this warning helped me discover several mistakes that would have been very difficult to track down without it. The associated pull request is [here](https://bitbucket.org/eschnett/carpet/pull-requests/53/carpet-added-w…. For example, it made me realize that I never properly changed over BaikalVacuum’s BC selection to also register boundary conditions with the driver. If I had put this comment in earlier, my confusion about why the code was trying to sync\+apply BCs way too often would have been instantly solved. As another example, one function in BaikalVacuum reads one group everywhere and another interior. I had both everywhere, but the interior-only one has no BCs because it is only ever read on the interior. As such, I was triggering syncs for no reason. This warning pointed me to the real problem in my simulation very quickly, and this was causing my code to sync variables 24 \(RK4 \* variables in group\) times more per iteration. Finally, it led to me finding that MoL’s scheduling was trying to sync scalars \(see [Ticket #2650](https://bitbucket.org/einsteintoolkit/tickets/issues/2650/fixing-write-statements-for-mol-scalars)\). While not a performance issue like the above, this is incorrect behavior. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2651/new-warning-messa…

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Trac