[Users] boundary conditions with Llama

[Miguel Zilhão]

hi Roland, all,

i'm coming back to this point since i'm now realizing that my interpretation was not correct... just 
to recall, i'm imposing boundary conditions in the following fashion:

    reflevel = GetRefinementLevel(cctkGH)
    map      = MultiPatch_GetMap(cctkGH)
    if (reflevel /= 0 .or. map == 0) return

    do j = 1, cctk_lsh(2)
       do i = 1, cctk_lsh(1)
          do k = cctk_lsh(3)-cctk_nghostzones(3)+1, cctk_lsh(3)
             if (Sn(i,j,k) == -2) then

               [my BCs go here]

             end if
          end do
       end do
    end do

and my assumption was that the line "if (Sn(i,j,k) == -2)" would guarantee that this would only run 
in the physical boundary region, as per the comment in the file interface.ccl from llama/Interpolate2:

   CCTK_INT source_patch TYPE=gf TAGS='Checkpoint="no" Prolongation="none"'
   {
     Sn
   } "source patch number, -1 for interior points, -2 for outer boundary points, -3 for inter-
processor ghost points"

this interpretation was consistent with what it's done in the thorn LlamaWaveToy (where the BC are 
imposed in a similar way).

however, when running with several processors, it seems that there are some inter-processor points 
marked with Sn = -2, and therefore the loop above is called outside my physical boundary region (i 
noticed this by adding write statements inside the loop above).

so i have the following questions:

  - what exactly does "outer boundary points" mean for the Sn function? is there a bug, or did i 
just misinterpreted that "outer boundary" means the "physical" boundary?

  - what would be the canonical way of checking whether a point belongs to the physical outer 
boundary (ie, where BCs should be specified) when using Llama?

thanks,
Miguel


On 18/06/19 16:00, Haas, Roland wrote:
> Hello Miguel,
> 
>> oh, but won't this be handled by the
>>
>>     if (Sn(i,j,k) == -2)
>>
>> line above?
> I see. I had no idea that this is what the grid function does.
> 
>> from the description of the Sn function in the Interpolate2 thorn:
>>
>>    CCTK_INT source_patch TYPE=gf TAGS='Checkpoint="no"
>> Prolongation="none"' {
>>      Sn
>>    } "source patch number, -1 for interior points, -2 for outer
>> boundary points, -3 for inter- processor ghost points"
>>
>> and also from the way this is done in LlamaWaveToy, i was assuming
>> that Sn(i,j,k) == -2 would guarantee that the points belong to the
>> outer boundary (ie, points where i want to specify a physical
>> boundary condition). is this assumption not correct?
> Yes, in that case this should work fine. In fact it is probably more
> correct then what I did b/c due to the way Llama works may suggestion
> would likely have treated the inter-patch bondaries in Llama as outer
> boundaries (since they are "symmetry" boundaries to Cactus).
> 
> So you should be fine actually.
> 
> Yours,
> Roland
>