[Users] Llama - Interpatch boundaries found

[Samuel Tootle]

Hi Erik,

Thanks for taking the time to reply.

I had a chance to come back to this and was able to resolve the issue 
which came down to how I was rescaling the problem to larger mass 
systems (Mtot != 1) and mass ratios q < 1.

The trouble region, as I found in the GW150914 parfile, is related to 
Coordinates::sphere_inner_radius

Where the integer multiple of h0 to compute sphere_inner_radius was 
becoming too small.  Setting a floor to this integer multiple of 40 
resolved the issue for now.

`sphere_inner_radius = max(int(ceil(sphere_inner_radius/(i*h0))) * i, 
40) * h0`

Cheers,

Samuel

On 1/17/23 7:15 PM, Erik Schnetter wrote:
> Samuel
>
> Carpet is adding buffer and ghost zones to each refinement level. This
> might enlarge certain levels by quite a bit, especially when the
> resolution is low. Coarser levels then need to be enlarged as well,
> and will in turn have their buffer and ghost zones added. This can
> quickly make refined regions much larger than you expect.
>
> To experiment, you can switch off buffer and ghost zones when setting
> up the grid structure, and output the grid structure to examine it
> e.g. in VisIt. You can also increase the resolution to see whether
> this helps. For these tests you do not need to activate any physics
> thorns (ML_BSSN, ADMBase, etc.), and doing so can reduce your memory
> consumption during these tests.
>
> -erik
>
>
>
>
> On Mon, Jan 16, 2023 at 11:30 AM Samuel Tootle
> <tootle at itp.uni-frankfurt.de> wrote:
>> All:
>>
>> I've been playing with the GW150914 gallery file and testing the setup
>> with different mass ratio BBH initial data given how fast the
>> simulations are using the multipatch setup.  I made some modifications
>> to the gallery script to match my setup (the final par is attached),
>> however, I keep getting an odd error:
>>
>>> INFO (Coordinates): Setting up global coordinates for thornburg04.
>>> INFO (Coordinates): Setting local coordinates: Thornburg04
>>> WARNING level 0 from host r36c1t1n1.ib0.hawk.hww.hlrs.de process 0
>>>    while executing schedule bin Interpolate2Test, routine
>>> Interpolate2::Interpolate2TestInit
>>>    in thorn Interpolate2, file
>>> /zhome/academic/HLRS/xfp/xfpstoot/lib/FreeTHC2020_new/Cactus/arrangements/Llama/Interpolate2/src/test.cc:77:
>>>    -> Refinement level 1 contains inter-patch boundaries; this is
>>> currently not supported
>> The current setup is looking at a BBH setup with q = 4.  I had the same
>> problem with q=2 as it seemed some of the refinement levels from the
>> primary came close to one of those from the secondary so I staggered
>> them more by making the refinement levels for the primary increase by
>> factors of two and a factor of 1.5 for the secondary.  However, I am
>> getting the same error even when using this new staggering setup.
>>
>> Looking at the grid output (see run.log) I don't see where this
>> "inter-patch" issue is showing up and increasing verbosity didn't shed
>> further light from my optic.  Before dedicating more time, I wanted to
>> see if maybe the more experienced with these thorns might have some
>> insight as to what the problem may be.
>>
>> Cheers,
>>
>> Samuel
>> _______________________________________________
>> Users mailing list
>> Users at einsteintoolkit.org
>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
>