[Users] CarpetMask problem (was: Re: [Commits] [svn:einsteintoolkit] Paper_EinsteinToolkit_2010/examples/bbh/ (Rev. 85))

[Erik Schnetter]

Bruno

The problem was indeed that CarpetReduce overwrote the mask. While
setting up the mask, CarpetReduce uses a temporary integer grid
function to store which octant of which cell is active. This is later
converted into the real grid function that people use. CarpetMask
needed to set up this integer grid function instead of the real grid
function.

Please update Carpet and try again.

-erik

On Wed, May 11, 2011 at 6:39 PM, Bruno Coutinho Mundim
<bcmsma at astro.rit.edu> wrote:
> Hi Erik,
>
> I have attached the schedule output for this run. I suspect CarpetReduce
> is somehow overwriting the weight function set by CarpetMask.
>
> Thanks,
> Bruno.
>
> Erik Schnetter wrote:
>>
>> That is strange. I assume some other routine overwrites it. Can you
>> check the schedule? Is there another CarpetMask or CarpetReduce
>> routine acting on the mask afterwards?
>>
>> -erik
>>
>> On Wed, May 11, 2011 at 1:30 PM, Bruno C. Mundim <bcmsma at astro.rit.edu>
>> wrote:
>>>
>>> Thanks, Erik! On a related note: I checked if CarpetMask was setting
>>> weight[ind]=0 inside the horizons and it seems to do so correctly.
>>> However CarpetReduce::weight still do not reflect that. Its output
>>> still doesn't have the horizons masked out.
>>>
>>> Thanks,
>>> Bruno.
>>>
>>> Erik Schnetter wrote:
>>>>
>>>> On Wed, May 11, 2011 at 12:53 PM, Bruno C. Mundim <bcmsma at astro.rit.edu>
>>>> wrote:
>>>>>
>>>>> Hi Roland:
>>>>>
>>>>> Roland Haas wrote:
>>>>>>
>>>>>> Hello Bruno,
>>>>>>
>>>>>>>  a) Its evolution is stopped with Nans around 8.6M. Nans appear to
>>>>>>> come
>>>>>>>  from the BSSN fields, but still needs further investigation.
>>>>>>
>>>>>> Hmm I could evolve the qc0 example in McLachlan. Maybe there are
>>>>>> differences in the parameter file that make the error "obvious".
>>>>>>
>>>>> A few obvious changes that I can think of:
>>>>>
>>>>> 1) I reduced the number of spherical surfaces from 5 to 3. Now the
>>>>> puncture tracker uses surfaces 0 and 1, AHFinderDirect track those
>>>>> surfaces and load AH related info there. QLM, CarpetTracker and
>>>>> CarpetMask uses those 2 surfaces then. I will go back to 5 surfaces
>>>>> as it was set before. Maybe I am missing some subtle scheduling order
>>>>> here. I don't know.
>>>>>
>>>>> 2) I added a few general parameters I am used to use to CarpetInterp,
>>>>> CarpetLib and CarpetRegrid2.
>>>>>
>>>>> 3) From a discussion I had with Peter on the ET workshop, I decided
>>>>> to change the AHFinderDirect interpolator to Hermite.
>>>>>
>>>>> So I will switch back to what it was and avoid to do so many changes
>>>>> at once! :) However I find strange that these changes could be the
>>>>> cause of what I am seeing.
>>>>>
>>>>>
>>>>>>>  c) When CarpetReduce::weight is output, its values seem strange to
>>>>>>> me.
>>>>>>>  Most of the domain has 1/2 set. A few points close to the boundary
>>>>>>> 1/4
>>>>>>>  and the boundaries 0 (physical and ghost). No mask out values inside
>>>>>>>  AH is shown, ie they still are set to 1/2. These values seem to make
>>>>>>>  more sense to AMR restriction operations.
>>>>>>
>>>>>> Are you maybe looking at the z=0 plane? That plane has a weight of 1/2
>>>>>> since only half of the volume of each cell is used when eg. computing
>>>>>> the simulation volume. I have never used CarpetMask myself (but have
>>>>>> written a similar thorn of my own... would have paid to look into
>>>>>> Carpet
>>>>>> first I guess :-) )
>>>>>>
>>>>> Yes I was looking at the z=0 plane and had refection symmetry set up!
>>>>> Thanks for pointing this out. If we go out of the plane then the
>>>>> weight=1.0 values show up. A more careful look into the data, I noticed
>>>>> the zeros are set in the buffer zones indeed with the exception of the
>>>>> third point (ghostzone=3) in the domain (from the AMR boundary) that
>>>>> has
>>>>> its value set to 1/4. Do you know why?
>>>>
>>>> For some reason that I currently cannot recall, the ghost zones in the
>>>> mask may be set to 1 instead of 0. However, ghost zones are ignored
>>>> anyway by a different mechanism when reducing (by using cctk_bbox and
>>>> cctk_nghostzones).
>>>>
>>>> The particular values indicate how much of the "control cell"
>>>> surrounding this grid point should be taken into account when
>>>> reducing. If the value is 1/4, then you may be looking at an edge
>>>> point (e.g. and x-y edge in the z=0 plane), where 3/4 of the control
>>>> cell are already covered by the corresponding coarse grid.
>>>>
>>>> One important test is that the sum of all weights times the grid cell
>>>> volume equals the total volume of the simulation domain. If you look
>>>> into the simulation log output and search for "volume" there, you
>>>> should find a corresponding statement.
>>>>
>>>> -erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/