[Users] CarpetMask problem (was: Re: [Commits] [svn:einsteintoolkit] Paper_EinsteinToolkit_2010/examples/bbh/ (Rev. 85))

Thu May 12 10:09:02 CDT 2011

Bruno

Ah, of course, yes. Yes, this message should disappear if you don't
mask out the interior any more, and the volume difference should be
the AH coordinate volume.

-erik

On Thu, May 12, 2011 at 2:13 AM, Bruno C. Mundim <bcmsma at astro.rit.edu> wrote:
> Thanks, Erik! The mask now seems to work appropriately. I have one more
> question: CarpetReduce is not outputting the following message:
>
> INFO (CarpetReduce): Simulation domain volume: 432000
> INFO (CarpetReduce): Reduction weight sum:     431999.99694252
> WARNING level 1 in thorn CarpetReduce processor 0 host
> i115-201.ranger.tacc.utexas.edu
>  (line 120 of
> /work/01157/bcmundim/einstein_dev_curie/Cactus/arrangements/Carpet/CarpetReduce/src/mask_test.c):
>  -> Simulation domain volume and reduction weight sum differ
>
> Should I be concerned about this? or is it due to the fact that the AH
> interior is masked out?
>
> Thanks,
> Bruno.
>
>
> Erik Schnetter wrote:
>>
>> Bruno
>>
>> The problem was indeed that CarpetReduce overwrote the mask. While
>> setting up the mask, CarpetReduce uses a temporary integer grid
>> function to store which octant of which cell is active. This is later
>> converted into the real grid function that people use. CarpetMask
>> needed to set up this integer grid function instead of the real grid
>> function.
>>
>> Please update Carpet and try again.
>>
>> -erik
>>
>> On Wed, May 11, 2011 at 6:39 PM, Bruno Coutinho Mundim
>> <bcmsma at astro.rit.edu> wrote:
>>>
>>> Hi Erik,
>>>
>>> I have attached the schedule output for this run. I suspect CarpetReduce
>>> is somehow overwriting the weight function set by CarpetMask.
>>>
>>> Thanks,
>>> Bruno.
>>>
>>> Erik Schnetter wrote:
>>>>
>>>> That is strange. I assume some other routine overwrites it. Can you
>>>> check the schedule? Is there another CarpetMask or CarpetReduce
>>>> routine acting on the mask afterwards?
>>>>
>>>> -erik
>>>>
>>>> On Wed, May 11, 2011 at 1:30 PM, Bruno C. Mundim <bcmsma at astro.rit.edu>
>>>> wrote:
>>>>>
>>>>> Thanks, Erik! On a related note: I checked if CarpetMask was setting
>>>>> weight[ind]=0 inside the horizons and it seems to do so correctly.
>>>>> However CarpetReduce::weight still do not reflect that. Its output
>>>>> still doesn't have the horizons masked out.
>>>>>
>>>>> Thanks,
>>>>> Bruno.
>>>>>
>>>>> Erik Schnetter wrote:
>>>>>>
>>>>>> On Wed, May 11, 2011 at 12:53 PM, Bruno C. Mundim
>>>>>> <bcmsma at astro.rit.edu>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Roland:
>>>>>>>
>>>>>>> Roland Haas wrote:
>>>>>>>>
>>>>>>>> Hello Bruno,
>>>>>>>>
>>>>>>>>>  a) Its evolution is stopped with Nans around 8.6M. Nans appear to
>>>>>>>>> come
>>>>>>>>>  from the BSSN fields, but still needs further investigation.
>>>>>>>>
>>>>>>>> Hmm I could evolve the qc0 example in McLachlan. Maybe there are
>>>>>>>> differences in the parameter file that make the error "obvious".
>>>>>>>>
>>>>>>> A few obvious changes that I can think of:
>>>>>>>
>>>>>>> 1) I reduced the number of spherical surfaces from 5 to 3. Now the
>>>>>>> puncture tracker uses surfaces 0 and 1, AHFinderDirect track those
>>>>>>> surfaces and load AH related info there. QLM, CarpetTracker and
>>>>>>> CarpetMask uses those 2 surfaces then. I will go back to 5 surfaces
>>>>>>> as it was set before. Maybe I am missing some subtle scheduling order
>>>>>>> here. I don't know.
>>>>>>>
>>>>>>> 2) I added a few general parameters I am used to use to CarpetInterp,
>>>>>>> CarpetLib and CarpetRegrid2.
>>>>>>>
>>>>>>> 3) From a discussion I had with Peter on the ET workshop, I decided
>>>>>>> to change the AHFinderDirect interpolator to Hermite.
>>>>>>>
>>>>>>> So I will switch back to what it was and avoid to do so many changes
>>>>>>> at once! :) However I find strange that these changes could be the
>>>>>>> cause of what I am seeing.
>>>>>>>
>>>>>>>
>>>>>>>>>  c) When CarpetReduce::weight is output, its values seem strange to
>>>>>>>>> me.
>>>>>>>>>  Most of the domain has 1/2 set. A few points close to the boundary
>>>>>>>>> 1/4
>>>>>>>>>  and the boundaries 0 (physical and ghost). No mask out values
>>>>>>>>> inside
>>>>>>>>>  AH is shown, ie they still are set to 1/2. These values seem to
>>>>>>>>> make
>>>>>>>>>  more sense to AMR restriction operations.
>>>>>>>>
>>>>>>>> Are you maybe looking at the z=0 plane? That plane has a weight of
>>>>>>>> 1/2
>>>>>>>> since only half of the volume of each cell is used when eg.
>>>>>>>> computing
>>>>>>>> the simulation volume. I have never used CarpetMask myself (but have
>>>>>>>> written a similar thorn of my own... would have paid to look into
>>>>>>>> Carpet
>>>>>>>> first I guess :-) )
>>>>>>>>
>>>>>>> Yes I was looking at the z=0 plane and had refection symmetry set up!
>>>>>>> Thanks for pointing this out. If we go out of the plane then the
>>>>>>> weight=1.0 values show up. A more careful look into the data, I
>>>>>>> noticed
>>>>>>> the zeros are set in the buffer zones indeed with the exception of
>>>>>>> the
>>>>>>> third point (ghostzone=3) in the domain (from the AMR boundary) that
>>>>>>> has
>>>>>>> its value set to 1/4. Do you know why?
>>>>>>
>>>>>> For some reason that I currently cannot recall, the ghost zones in the
>>>>>> mask may be set to 1 instead of 0. However, ghost zones are ignored
>>>>>> anyway by a different mechanism when reducing (by using cctk_bbox and
>>>>>> cctk_nghostzones).
>>>>>>
>>>>>> The particular values indicate how much of the "control cell"
>>>>>> surrounding this grid point should be taken into account when
>>>>>> reducing. If the value is 1/4, then you may be looking at an edge
>>>>>> point (e.g. and x-y edge in the z=0 plane), where 3/4 of the control
>>>>>> cell are already covered by the corresponding coarse grid.
>>>>>>
>>>>>> One important test is that the sum of all weights times the grid cell
>>>>>> volume equals the total volume of the simulation domain. If you look
>>>>>> into the simulation log output and search for "volume" there, you
>>>>>> should find a corresponding statement.
>>>>>>
>>>>>> -erik
>>
>

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/