Trac

trac@lists.einsteintoolkit.org

8708 discussions

#2648: Empty ApplyBCs call in LeanBSSNMoL
by helvi witek 27 Jan '23

27 Jan '23

#2648: Empty ApplyBCs call in LeanBSSNMoL Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: major Component: EinsteinToolkit thorn Comment (by helvi witek): And the conversation between Roland and myself: **hwitek** [3:40 PM](https://nrfunphyswitekgroup.slack.com/archives/C03RAQN3H1R/p16747692255… actually Sam had contacted me directly via email, but yes this should be in a ticket. [3:43](https://nrfunphyswitekgroup.slack.com/archives/C03RAQN3H1R/p167476939… The original reason was to apply **inner** $refinement or ghost zone$ boundary condition with the Cactus/Carpet “ApplyBCs” function, directly after computing the bssn variables, including $\\Gamma^\{i\}$. It is not clear to me that “ApplyBCs” really does nothing, and am therefore hesitant to just take it out without it having been tested for a non-trivial case. **Roland Haas** [9:39 AM](https://nrfunphyswitekgroup.slack.com/archives/C03RAQN3H1R/p16748339565… so many avenues of interaction. The scheduled functions in the ApplyBCs group will not do anything unless there has been a SelectGroupForBCs first though $and no intervening ApplyBCs which clears the selections$. SelectGroupForBCs does not have to be immediatedly before the ApplyBCs group though, so there could be a "forgotten" / "hidden" one somewhere earlier in the code. Also if this is during initial data only $and I really should do as I say and use the ticket$... then this may not be very visible at all if the initial data thorn did fill in data everywhere on all grid points irrespective of them being ghosts, buffer, physical boundary, symmetry boundary or interior points. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2648/empty-applybcs-ca…

1 0

#2648: Empty ApplyBCs call in LeanBSSNMoL
by helvi witek 27 Jan '23

27 Jan '23

#2648: Empty ApplyBCs call in LeanBSSNMoL Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: major Component: EinsteinToolkit thorn Comment (by helvi witek): So, for full disclosure let me copy here the email conversion with Sam, Miguel and myself $apologies for email formatting$: It sounds like you would want something like LeanBSSN\_Boundaries running before that. However, looking at that function I see quite a lot of grid functions in it. Do you want all of them to run? If you only want gammat you'll $unfortunately$ probably need another function with just it. However, it's probably fine to apply symmetry conditions to the other grid functions $gconf\_fac, hmetric, hcurv, trk, lapse, shift$. Again, I wouldn't support that if it was in the evolution loop, but a few extra symmetry BCs in initial shouldn't really matter for speed unless its a very short run. Also, keep in mind that I haven't actually put in any prints inside CartGrid3D and am going purely off of looking at the code. You may want to see if anything is actually being applied before changing the code or scheduling. I just thought it looked incorrect and thought you might want to check it out. I had started a [ticket](https://urldefense.com/v3/__https://bitbucket.org/einsteintoolkit/t… a while back, so we can use that if there's any changes you want to make or questions for the maintainers. Samuel Cupp Postdoctoral Researcher Department of Physics University of Idaho --- **From:** Miguel Zilhão <[mzilhao@ua.pt](mailto:mzilhao@ua.pt)> **Sent:** Saturday, January 21, 2023 3:05 AM **To:** Cupp, Samuel D. <[scupp1@my.apsu.edu](mailto:scupp1@my.apsu.edu)>; Helvi Witek <[hwitek@illinois.edu](mailto:hwitek@illinois.edu)> **Subject:** Re: LeanBSSNMoL scheduling hi Sam, thanks for your detailed reply. i understand better your point and i'm now a bit confused. let me elaborate below. > Thanks for the quick response. The ghost zones $inter-processor boundaries$ wouldn't be triggered by > this. They are triggered by > > schedule LeanBSSN\_adm2bssn at CCTK\_INITIAL after ADMBase\_PostInitial > \{ > LANG: Fortran > OPTIONS: Local > SYNC: gammat > \} "Convert initial data into BSSN variables" > > where the \*SYNC\* statement is. indeed, you're totally right. > I asked Steve and Roland about when symmetry BCs are applied just to > make sure I understood correctly, and the symmetry BCs are applied whenever the physical BCs are > applied. However, physical BCs only get applied by the ApplyBCs function when a Select\*ForBC > function call has actually registered a function with the Boundary thorn. Since no Select\*ForBC > functions are scheduled before this ApplyBC call, it shouldn't do anything. ah, wait. my understanding was that we were registering the symmetry BCs in the following call in the schedule file: schedule LeanBSSN\_symmetries at BASEGRID \{ LANG: Fortran OPTIONS: Global \} "Register symmetries of the BSSN grid functions" called at "BASEGRID". inside the routine "LeanBSSN\_symmetries" we have call SetCartSymVN$ ierr, cctkGH, \(/ 1, 1, 1/$, "LeanBSSNMoL::conf\_fac" \) call SetCartSymVN$ ierr, cctkGH, \(/ 1, 1, 1/$, "LeanBSSNMoL::rhs\_conf\_fac" \) call SetCartSymVN$ ierr, cctkGH, \(/ 1, 1, 1/$, "LeanBSSNMoL::hxx" \) call SetCartSymVN$ ierr, cctkGH, \(/-1,-1, 1/$, "LeanBSSNMoL::hxy" \) call SetCartSymVN$ ierr, cctkGH, \(/-1, 1,-1/$, "LeanBSSNMoL::hxz" \) call SetCartSymVN$ ierr, cctkGH, \(/ 1, 1, 1/$, "LeanBSSNMoL::hyy" \) call SetCartSymVN$ ierr, cctkGH, \(/ 1,-1,-1/$, "LeanBSSNMoL::hyz" \) call SetCartSymVN$ ierr, cctkGH, \(/ 1, 1, 1/$, "LeanBSSNMoL::hzz" \) $...$ so what you're saying is that calling "ApplyBCs" after this is meaningless by itself unless one actually calls the Select\*ForBC functions before? indeed, it seems we only call Select\*ForBC much later $at MoL\_PostStep$. > Roland mentioned that > "Correct. If there is not SelectBCs call then nothing will happen. It \*is\* possible that some of the > old-style symmetry boundary conditions from CartGrid3D still get applied, which has some code in > there $I think$ to work with thorn Boundary." it could be that there is something making sure that some symmetry conditions are applied, otherwise in every run with symmetry the gammat\* grid functions would have non-initialized values at the symmetry points, which would likely create problems... > So I looked to see how CartGrid3D triggers its BCs, and it also relies on the selected variables > from Boundary. So, best I can tell this does nothing. It isn't a real issue or anything, as its a > few empty function calls that happens once at INITIAL. Removing it wouldn't make the code faster or > anything, but I don't think it is needed. Alternatively, you may want a function SelectionVarForBC > that actually causes gammat to have symmetry BCs applied. i think this is probably the safest thing, just adding a line in the schedule file to make sure the "LeanBSSN\_Boundaries" function is also called after at CCTK\_INITIAL. what do you think? thanks, Miguel > ---------------------------------------------------------------------------------------------------- > \*From:\* Miguel Zilhão <[mzilhao@ua.pt](mailto:mzilhao@ua.pt)> > \*Sent:\* Friday, January 20, 2023 2:58 AM > \*To:\* Helvi Witek <[hwitek@illinois.edu](mailto:hwitek@illinois.edu)>; Cupp, Samuel D. <[scupp1@my.apsu.edu](mailto:scupp1@my.apsu.edu)> > \*Subject:\* Re: LeanBSSNMoL scheduling > hi Sam and Helvi, > > if i remember correctly, that call to "ApplyBCs" is there since in the routine > LeanBSSN\_adm2bssn we're not setting the values of the grid functions > gammat\{x,y,z\} at the ghostzones $since this is done via some derivatives$. we > then call "ExtrapolateGammas" on these functions to set the values at the outer > boundary, but the inter-processor and symmetry boundaries will remain unset. > > with the call to "ApplyBCs", inter-processor and symmetry boundaries are > applied, and the gammat\* grid functions are then defined everywhere. > > this was the reasoning, i think. does it make sense? > > cheers, > Miguel > > On Thu, 2023-01-19 at 13:25 -0600, Helvi Witek wrote: >> Dear Sam, >> let me also cc Miguel. >> >> As far as I recall, this boundary scheduling is the same as for McLachlan. It >> is done right after the conversion from adm to bssn $including computing the >> conformal connection function$. >> >> @Miguel, do you recall the details of the "ApplyBCs" call here? >> >> Best wishes, >> Helvi >> >> --------------------------------------------- >> Dr. Helvi Witek >> Assistant Professor >> Department of Physics >> University of Illinois at Urbana-Champaign >> >> 247 Loomis Lab >> 1110 W Green St >> Urbana, IL 61801 >> --------------------------------------------- >> On 1/19/23 5:49 PM, Cupp, Samuel D. wrote: >> >> > >> > Hey Helvi, >> > I'm wondering if maybe my previous email went to your junk. A while ago >> > I had asked about LeanBSSNMoL. I'll just copy the previous email into this >> > one. >> > >> > I have been working on Baikal and BaikalVacuum $in wvuthorns$, and they >> > copied a lot of their scheduling from LeanBSSNMoL. In doing so, I noticed a >> > weird scheduled group in the schedule.ccl that is also in LeanBSSNMoL. I was >> > hoping you or someone else who worked on Lean could explain why it is there. >> > >> > In LeanBSSNMoL's schedule.ccl is >> > >> > schedule GROUP ApplyBCs as LeanBSSN\_ApplyBCs at CCTK\_INITIAL after LeanBSSN\_ >> > adm2bssn >> > \{ >> > \} "Apply boundary conditions" >> > >> > However, there are no boundary condition selection routines which run >> > before this, and nothing is scheduled relative to this function. As best I >> > can tell, this should do nothing and could just be removed. Is there >> > something I'm missing? Or, should there be a SelectBCs routine running here >> > that is missing? Thanks for your help. >> > >> > Samuel Cupp Postdoctoral Researcher >> > Department of Physics >> > ‌ ‌ -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2648/empty-applybcs-ca…

1 0

#2648: Empty ApplyBCs call in LeanBSSNMoL
by Roland Haas 27 Jan '23

27 Jan '23

#2648: Empty ApplyBCs call in LeanBSSNMoL Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: major Component: EinsteinToolkit thorn Comment (by Roland Haas): Note: “major” bugs are those that affect results and must be fixed before the next release. If the maintainers of lean\_public and Baikal could look into this and adjust the severity setting accordingly that would be great. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2648/empty-applybcs-ca…

1 0

#2648: Empty ApplyBCs call in LeanBSSNMoL
by Roland Haas 27 Jan '23

27 Jan '23

#2648: Empty ApplyBCs call in LeanBSSNMoL Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: major Component: EinsteinToolkit thorn Changes (by Roland Haas): responsible: [] (was ) assignee: helvi witek (was ) -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2648/empty-applybcs-ca…

1 0

#2648: Empty ApplyBCs call in LeanBSSNMoL
by Roland Haas 27 Jan '23

27 Jan '23

#2648: Empty ApplyBCs call in LeanBSSNMoL Reporter: Samuel Cupp Status: new Milestone: Version: Type: bug Priority: major Component: EinsteinToolkit thorn Comment (by Roland Haas): The scheduled functions in the ApplyBCs group will not do anything unless there has been a SelectGroupForBCs first though $and no intervening ApplyBCs which clears the selections$. SelectGroupForBCs does not have to be immediatedly before the ApplyBCs group though, so there could be a "forgotten" / "hidden" one somewhere earlier in the code. Also if this is during initial data only then this may not be very visible at all if the initial data thorn did fill in data everywhere on all grid points irrespective of them being ghosts, buffer, physical boundary, symmetry boundary or interior points. After ADM2BSSN one usually has to SYNC, extrapolate and apply symmetry boundary conditions $possibly in a very specific order$ and the Gamma auxiliary variables of the BSSN system which involve derivatives so cannot be set in ghost zones and the boundary. Most others can be computed point by point, so that a missing SYNC $missing for sure it seems$ will only show up a slight change in regions filled via prolongation $but not in inter-processor ghosts or in restriction target region$. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2648/empty-applybcs-ca…

1 0

#2686: Rename CCTK_REAL -> double when float is not tested
by Erik Schnetter 26 Jan '23

26 Jan '23

#2686: Rename CCTK_REAL -> double when float is not tested Reporter: Zach Etienne Status: new Milestone: Version: Type: enhancement Priority: minor Component: Comment (by Erik Schnetter): See [https://en.cppreference.com/w/c/numeric/tgmath](https://en.cppreference.com… for C. $Fortran and C\+\+ already support this generic functionality.$ -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2686/rename-cctk_real-…

1 0

#2686: Rename CCTK_REAL -> double when float is not tested
by Zach Etienne 26 Jan '23

26 Jan '23

#2686: Rename CCTK_REAL -> double when float is not tested Reporter: Zach Etienne Status: new Milestone: Version: Type: enhancement Priority: minor Component: Comment (by Zach Etienne): Roland pointed out during today’s call that the compile flag REAL\_PRECISION can be set to 4, 8, or 16 bytes. It’s likely this directly sets CCTK\_REAL. Also if we would truly like a say, physics thorn, to be cross-compatible with different CCTK\_REAL aliases, we’d probably also need aliases for transcendental functions so that e.g., CCTK\_SIN$x$ would alias to sin$x$ for `double` , sinf$x$ for `float` , and sinl$x$ for `long double`. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2686/rename-cctk_real-…

1 0

#2666: Testsuite system does not recognize "Cactus" thorn
by Roland Haas 26 Jan '23

26 Jan '23

#2666: Testsuite system does not recognize "Cactus" thorn Reporter: Roland Haas Status: new Milestone: Version: development version Type: bug Priority: major Component: Cactus Comment (by Roland Haas): Unless objected I will apply this after 2023-02-09. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2666/testsuite-system-…

1 0

#2686: Rename CCTK_REAL -> double when float is not tested
by Zach Etienne 25 Jan '23

25 Jan '23

#2686: Rename CCTK_REAL -> double when float is not tested Reporter: Zach Etienne Status: new Milestone: Version: Type: enhancement Priority: minor Component: Comment (by Zach Etienne): I like the idea of testing all the commonly used thorns with single precision, but I worry it’s going to take a long time to debug and fix all the issues. $I’d love to be proven wrong!$ Compiler errors are just the start I fear; consider all the algorithms with tolerances tuned to relative errors ~1e-15 like conservatives-to primitives solvers, AH finders, etc. Also I bet our more complex finite-difference-based and possibly finite-volume-based codes $BSSN & GRMHD codes$ will become unstable & unusable. But again, I’d love to be proven wrong. :slight_smile: -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2686/rename-cctk_real-…

1 0

#2686: Rename CCTK_REAL -> double when float is not tested
by Erik Schnetter 25 Jan '23

25 Jan '23

#2686: Rename CCTK_REAL -> double when float is not tested Reporter: Zach Etienne Status: new Milestone: Version: Type: enhancement Priority: minor Component: Comment (by Erik Schnetter): I’m not convinced they would break. I assume that many of them would switch to double precision for intermediate calculations, but they wouldn’t per se break. And if they break then it’s usually just a compile-time error that is straightforward to correct, such as e.g. calling `max(x, 1.0)` in C\+\+. In fact, I am going to propose that we should build the commonly used thorns with single precision and correct these compiler errors. -- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2686/rename-cctk_real-…

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Trac