[ET Trac] #2648: Empty ApplyBCs call in LeanBSSNMoL
helvi witek
trac-noreply at einsteintoolkit.org
Fri Jan 27 09:35:45 CST 2023
#2648: Empty ApplyBCs call in LeanBSSNMoL
Reporter: Samuel Cupp
Status: new
Milestone:
Version:
Type: bug
Priority: major
Component: EinsteinToolkit thorn
Comment (by helvi witek):
So, for full disclosure let me copy here the email conversion with Sam, Miguel and myself \(apologies for email formatting\):
It sounds like you would want something like LeanBSSN\_Boundaries running before that. However, looking at that function I see quite a lot of grid functions in it. Do you want all of them to run? If you only want gammat you'll \(unfortunately\) probably need another function with just it. However, it's probably fine to apply symmetry conditions to the other grid functions \(gconf\_fac, hmetric, hcurv, trk, lapse, shift\). Again, I wouldn't support that if it was in the evolution loop, but a few extra symmetry BCs in initial shouldn't really matter for speed unless its a very short run.
Also, keep in mind that I haven't actually put in any prints inside CartGrid3D and am going purely off of looking at the code. You may want to see if anything is actually being applied before changing the code or scheduling. I just thought it looked incorrect and thought you might want to check it out.
I had started a [ticket](https://urldefense.com/v3/__https://bitbucket.org/einsteintoolkit/tickets/issues/2648/empty-applybcs-call-in-leanbssnmol__;!!DZ3fjg!6P26S95OCIvFzHEpeRaPwr1L--HAnDJ1YZkznnTYju2mUnmF4BFyhBSMC27VrFe5KT2fJdg7FHAtOhwpIIPr$) a while back, so we can use that if there's any changes you want to make or questions for the maintainers.
Samuel Cupp Postdoctoral Researcher
Department of Physics University of Idaho
---
**From:** Miguel Zilhão <[mzilhao at ua.pt](mailto:mzilhao at ua.pt)>
**Sent:** Saturday, January 21, 2023 3:05 AM
**To:** Cupp, Samuel D. <[scupp1 at my.apsu.edu](mailto:scupp1 at my.apsu.edu)>; Helvi Witek <[hwitek at illinois.edu](mailto:hwitek at illinois.edu)>
**Subject:** Re: LeanBSSNMoL scheduling
hi Sam,
thanks for your detailed reply. i understand better your point and i'm now a bit confused. let me
elaborate below.
> Thanks for the quick response. The ghost zones \(inter-processor boundaries\) wouldn't be triggered by
> this. They are triggered by
>
> schedule LeanBSSN\_adm2bssn at CCTK\_INITIAL after ADMBase\_PostInitial
> \{
> LANG: Fortran
> OPTIONS: Local
> SYNC: gammat
> \} "Convert initial data into BSSN variables"
>
> where the \*SYNC\* statement is.
indeed, you're totally right.
> I asked Steve and Roland about when symmetry BCs are applied just to
> make sure I understood correctly, and the symmetry BCs are applied whenever the physical BCs are
> applied. However, physical BCs only get applied by the ApplyBCs function when a Select\*ForBC
> function call has actually registered a function with the Boundary thorn. Since no Select\*ForBC
> functions are scheduled before this ApplyBC call, it shouldn't do anything.
ah, wait. my understanding was that we were registering the symmetry BCs in the following call in
the schedule file:
schedule LeanBSSN\_symmetries at BASEGRID
\{
LANG: Fortran
OPTIONS: Global
\} "Register symmetries of the BSSN grid functions"
called at "BASEGRID". inside the routine "LeanBSSN\_symmetries" we have
call SetCartSymVN\( ierr, cctkGH, \(/ 1, 1, 1/\), "LeanBSSNMoL::conf\_fac" \)
call SetCartSymVN\( ierr, cctkGH, \(/ 1, 1, 1/\), "LeanBSSNMoL::rhs\_conf\_fac" \)
call SetCartSymVN\( ierr, cctkGH, \(/ 1, 1, 1/\), "LeanBSSNMoL::hxx" \)
call SetCartSymVN\( ierr, cctkGH, \(/-1,-1, 1/\), "LeanBSSNMoL::hxy" \)
call SetCartSymVN\( ierr, cctkGH, \(/-1, 1,-1/\), "LeanBSSNMoL::hxz" \)
call SetCartSymVN\( ierr, cctkGH, \(/ 1, 1, 1/\), "LeanBSSNMoL::hyy" \)
call SetCartSymVN\( ierr, cctkGH, \(/ 1,-1,-1/\), "LeanBSSNMoL::hyz" \)
call SetCartSymVN\( ierr, cctkGH, \(/ 1, 1, 1/\), "LeanBSSNMoL::hzz" \)
\(...\)
so what you're saying is that calling "ApplyBCs" after this is meaningless by itself unless one
actually calls the Select\*ForBC functions before? indeed, it seems we only call Select\*ForBC much
later \(at MoL\_PostStep\).
> Roland mentioned that
> "Correct. If there is not SelectBCs call then nothing will happen. It \*is\* possible that some of the
> old-style symmetry boundary conditions from CartGrid3D still get applied, which has some code in
> there \(I think\) to work with thorn Boundary."
it could be that there is something making sure that some symmetry conditions are applied, otherwise
in every run with symmetry the gammat\* grid functions would have non-initialized values at the
symmetry points, which would likely create problems...
> So I looked to see how CartGrid3D triggers its BCs, and it also relies on the selected variables
> from Boundary. So, best I can tell this does nothing. It isn't a real issue or anything, as its a
> few empty function calls that happens once at INITIAL. Removing it wouldn't make the code faster or
> anything, but I don't think it is needed. Alternatively, you may want a function SelectionVarForBC
> that actually causes gammat to have symmetry BCs applied.
i think this is probably the safest thing, just adding a line in the schedule file to make sure the
"LeanBSSN\_Boundaries" function is also called after at CCTK\_INITIAL. what do you think?
thanks,
Miguel
> ----------------------------------------------------------------------------------------------------
> \*From:\* Miguel Zilhão <[mzilhao at ua.pt](mailto:mzilhao at ua.pt)>
> \*Sent:\* Friday, January 20, 2023 2:58 AM
> \*To:\* Helvi Witek <[hwitek at illinois.edu](mailto:hwitek at illinois.edu)>; Cupp, Samuel D. <[scupp1 at my.apsu.edu](mailto:scupp1 at my.apsu.edu)>
> \*Subject:\* Re: LeanBSSNMoL scheduling
> hi Sam and Helvi,
>
> if i remember correctly, that call to "ApplyBCs" is there since in the routine
> LeanBSSN\_adm2bssn we're not setting the values of the grid functions
> gammat\{x,y,z\} at the ghostzones \(since this is done via some derivatives\). we
> then call "ExtrapolateGammas" on these functions to set the values at the outer
> boundary, but the inter-processor and symmetry boundaries will remain unset.
>
> with the call to "ApplyBCs", inter-processor and symmetry boundaries are
> applied, and the gammat\* grid functions are then defined everywhere.
>
> this was the reasoning, i think. does it make sense?
>
> cheers,
> Miguel
>
> On Thu, 2023-01-19 at 13:25 -0600, Helvi Witek wrote:
>> Dear Sam,
>> let me also cc Miguel.
>>
>> As far as I recall, this boundary scheduling is the same as for McLachlan. It
>> is done right after the conversion from adm to bssn \(including computing the
>> conformal connection function\).
>>
>> @Miguel, do you recall the details of the "ApplyBCs" call here?
>>
>> Best wishes,
>> Helvi
>>
>> ---------------------------------------------
>> Dr. Helvi Witek
>> Assistant Professor
>> Department of Physics
>> University of Illinois at Urbana-Champaign
>>
>> 247 Loomis Lab
>> 1110 W Green St
>> Urbana, IL 61801
>> ---------------------------------------------
>> On 1/19/23 5:49 PM, Cupp, Samuel D. wrote:
>>
>> >
>> > Hey Helvi,
>> > I'm wondering if maybe my previous email went to your junk. A while ago
>> > I had asked about LeanBSSNMoL. I'll just copy the previous email into this
>> > one.
>> >
>> > I have been working on Baikal and BaikalVacuum \(in wvuthorns\), and they
>> > copied a lot of their scheduling from LeanBSSNMoL. In doing so, I noticed a
>> > weird scheduled group in the schedule.ccl that is also in LeanBSSNMoL. I was
>> > hoping you or someone else who worked on Lean could explain why it is there.
>> >
>> > In LeanBSSNMoL's schedule.ccl is
>> >
>> > schedule GROUP ApplyBCs as LeanBSSN\_ApplyBCs at CCTK\_INITIAL after LeanBSSN\_
>> > adm2bssn
>> > \{
>> > \} "Apply boundary conditions"
>> >
>> > However, there are no boundary condition selection routines which run
>> > before this, and nothing is scheduled relative to this function. As best I
>> > can tell, this should do nothing and could just be removed. Is there
>> > something I'm missing? Or, should there be a SelectBCs routine running here
>> > that is missing? Thanks for your help.
>> >
>> > Samuel Cupp Postdoctoral Researcher
>> > Department of Physics
>> >
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2648/empty-applybcs-call-in-leanbssnmol
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/trac/attachments/20230127/dc0088e4/attachment-0001.html
More information about the Trac
mailing list