[Users] boundary conditions with Llama

[Erik Schnetter]

Miguel

Grid functions have edges and corners, and thus there are points that
are both "outer boundary" and "inter-process boundary".

The canonical way to find out where the outer boundaries are is to call

# The location of the boundary points
CCTK_INT FUNCTION MultiPatch_GetBoundarySpecification \
  (CCTK_INT IN map, \
   CCTK_INT IN size, \
   CCTK_INT OUT ARRAY nboundaryzones, \
   CCTK_INT OUT ARRAY is_internal, \
   CCTK_INT OUT ARRAY is_staggered, \
   CCTK_INT OUT ARRAY shiftout)

This specifies, for a given map, how many boundary points there are
(nboundaryzones).

You also call

CCTK_INT FUNCTION                         \
    MultiPatch_GetBbox                    \
        (CCTK_POINTER_TO_CONST IN cctkGH, \
         CCTK_INT IN size,                \
         CCTK_INT OUT ARRAY bbox)

to find out whether each of the 6 boundaries are outer boundaries.
"Outer boundary" here means that you need to apply an outer boundary
condition. Note that this might include some ghost zones. Depending on
your setup, you can either synchronize before applying outer boundary
conditions, and then apply them on ghost zones as well, or you can
skip applying outer boundary conditions on outer boundary points that
are ghost points, and then synchronize later.

To find out which points are ghost points, look at cctk_bbox in cctkGH.

-erik


On Tue, Jun 25, 2019 at 9:50 AM Haas, Roland <rhaas at illinois.edu> wrote:
>
> Hello Miguel,
>
> the official way would be to use CoordBase's GetBoundarySizesAndTypes
> function or replicate the code in repos/mclachlan/ML_BSSN/src/Kranc.cc,
> specifically GetBoundaryInfo.
>
> The repsonses in the is_symbnd from GetBoundarySizesAndTypes and
> GetBoundaryInfo likely differ since the former does not know about Llama so will classify Llama's inter-patch boundaries as "symmetry" boundaries while the latter does not do so.
>
> Their answer's for physical boundaries (which may not quite be the same
> as "outer" boundaries if symmetry boundaries count as "outer") should
> be identical though. And you are looking for the physical boundaries
> after all.
>
> Yours,
> Roland
>
> > hi Roland, all,
> >
> > i'm coming back to this point since i'm now realizing that my interpretation was not correct... just to recall, i'm imposing boundary conditions in the following fashion:
> >
> >     reflevel = GetRefinementLevel(cctkGH)
> >     map      = MultiPatch_GetMap(cctkGH)
> >     if (reflevel /= 0 .or. map == 0) return
> >
> >     do j = 1, cctk_lsh(2)
> >        do i = 1, cctk_lsh(1)
> >           do k = cctk_lsh(3)-cctk_nghostzones(3)+1, cctk_lsh(3)
> >              if (Sn(i,j,k) == -2) then
> >
> >                [my BCs go here]
> >
> >              end if
> >           end do
> >        end do
> >     end do
> >
> > and my assumption was that the line "if (Sn(i,j,k) == -2)" would guarantee that this would only run in the physical boundary region, as per the comment in the file interface.ccl from llama/Interpolate2:
> >
> >    CCTK_INT source_patch TYPE=gf TAGS='Checkpoint="no" Prolongation="none"'
> >    {
> >      Sn
> >    } "source patch number, -1 for interior points, -2 for outer boundary points, -3 for inter-
> > processor ghost points"
> >
> > this interpretation was consistent with what it's done in the thorn LlamaWaveToy (where the BC are imposed in a similar way).
> >
> > however, when running with several processors, it seems that there are some inter-processor points marked with Sn = -2, and therefore the loop above is called outside my physical boundary region (i noticed this by adding write statements inside the loop above).
> >
> > so i have the following questions:
> >
> >   - what exactly does "outer boundary points" mean for the Sn function? is there a bug, or did i just misinterpreted that "outer boundary" means the "physical" boundary?
> >
> >   - what would be the canonical way of checking whether a point belongs to the physical outer boundary (ie, where BCs should be specified) when using Llama?
> >
> > thanks,
> > Miguel
> >
> >
> > On 18/06/19 16:00, Haas, Roland wrote:
> > > Hello Miguel,
> > >
> > >> oh, but won't this be handled by the
> > >>
> > >>     if (Sn(i,j,k) == -2)
> > >>
> > >> line above?
> > > I see. I had no idea that this is what the grid function does.
> > >
> > >> from the description of the Sn function in the Interpolate2 thorn:
> > >>
> > >>    CCTK_INT source_patch TYPE=gf TAGS='Checkpoint="no"
> > >> Prolongation="none"' {
> > >>      Sn
> > >>    } "source patch number, -1 for interior points, -2 for outer
> > >> boundary points, -3 for inter- processor ghost points"
> > >>
> > >> and also from the way this is done in LlamaWaveToy, i was assuming
> > >> that Sn(i,j,k) == -2 would guarantee that the points belong to the
> > >> outer boundary (ie, points where i want to specify a physical
> > >> boundary condition). is this assumption not correct?
> > > Yes, in that case this should work fine. In fact it is probably more
> > > correct then what I did b/c due to the way Llama works may suggestion
> > > would likely have treated the inter-patch bondaries in Llama as outer
> > > boundaries (since they are "symmetry" boundaries to Cactus).
> > >
> > > So you should be fine actually.
> > >
> > > Yours,
> > > Roland
> > >
>
>
>
> --
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://pgp.mit.edu .
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users



-- 
Erik Schnetter <schnetter at cct.lsu.edu>
http://www.perimeterinstitute.ca/personal/eschnetter/