[Users] Problem with CarpetRegrid2/AMR

Hal Finkel hfinkel at anl.gov
Thu Dec 15 14:50:05 CST 2011


On Thu, 2011-12-15 at 15:45 -0500, Erik Schnetter wrote:
> Hal
> 
> 
> You should be able to apply boundary conditions (i.e. synchronised,
> apply symmetry conditions) in your code. You will also need to use
> sufficiently many processes to ensure that periodicity is applied
> correctly; this circumvents the bug/missing feature in the Periodic
> boundary conditions.

Erik,

Can you be more specific? And is a sufficient number just to make sure
that no one processor holds to opposing faces of the cube?

Thanks again,
Hal

> 
> 
> -erik
> 
> On Thu, Dec 15, 2011 at 3:14 PM, Hal Finkel <hfinkel at anl.gov> wrote:
>         On Thu, 2011-12-15 at 11:17 -0500, Erik Schnetter wrote:
>         > Eloisa
>         >
>         >
>         > Thanks for digging into this.
>         >
>         >
>         > I didn't think of syncing, but yes, syncing would be
>         necessary, and so
>         > would be setting all boundaries. In your case, this would be
>         applying
>         > periodic boundary conditions only (and not setting anything
>         to zero
>         > since you don't have an outer boundary). Of course, you need
>         to use
>         > sufficiently many processes for this because of the bug in
>         thorn
>         > Periodic.
>         
>         
>         Erik, Eloisa, et al.,
>         
>         Thank you very much for looking at this. Just so that I'm
>         clear, is
>         there a way that I can get this to work now, or am I waiting
>         on another
>         bug fix?
>         
>          -Hal
>         
>         >
>         >
>         > -erik
>         >
>         > On Thu, Dec 15, 2011 at 9:39 AM, Eloisa Bentivegna
>         > <bentivegna at cct.lsu.edu> wrote:
>         >         On Dec 15, 2011, at 5:58 AM, Hal Finkel wrote:
>         >
>         >         > On Wed, 2011-12-14 at 19:48 -0500, Erik Schnetter
>         wrote:
>         >         >> On Wed, Dec 14, 2011 at 6:53 PM, Hal Finkel
>         >         <hfinkel at anl.gov> wrote:
>         >         >>        On Wed, 2011-12-14 at 18:27 -0500, Erik
>         Schnetter
>         >         wrote:
>         >         >>> You are not setting any values on the boundary.
>         Is that
>         >         >>        intentional?
>         >         >>
>         >         >>
>         >         >>        Currently, this is because I am using
>         neighboring
>         >         values in
>         >         >>        the
>         >         >>        calculation, so I can't do that on the
>         boundary.
>         >         Should I do
>         >         >>        that some
>         >         >>        other way?
>         >         >>
>         >         >>
>         >         >> Setting the boundary to zero should be good
>         enough. (The
>         >         boundary
>         >         >> values should not be used -- but I don't recall
>         whether
>         >         this is the
>         >         >> case.)
>         >         >
>         >         > Maybe this is another problem with the periodic
>         boundary
>         >         conditions?
>         >         > The problem seems to appear in other fields too.
>         I've
>         >         attached some
>         >         > images from my test problem. One shows the field
>         >         configuration (this one
>         >         > looks like a ring -- it is a slice through  a
>         bubble). The
>         >         second one
>         >         > shows the T_00 computed from that. As you can see,
>         the
>         >         values near the
>         >         > extremal indicies are wrong. At the next (half)
>         time step,
>         >         looking at
>         >         > the field data from level 1, the same phenomonon
>         can be seen
>         >         in the
>         >         > level 1 boxes.
>         >         >
>         >         > These were all done in Kranc, so I did not do any
>         real
>         >         coding myself ;)
>         >         >
>         >         > What do you think?
>         >
>         >
>         >         Hi all!
>         >
>         >         I believe that what Hal is doing with the allocation
>         of
>         >         level_mask is correct; what seems to be problematic
>         is setting
>         >         its value.
>         >
>         >         The problem is that this function cannot be
>         calculated
>         >         pointwise, and the way it's currently set leaves
>         some parts of
>         >         the grid uninitialized, which then leads to poison
>         and
>         >         ultimately to all sorts of weird behavior including
>         nans at
>         >         the boundaries, lack of regridding when expected,
>         and so on. I
>         >         wouldn't pay too much attention to all these
>         symptoms
>         >         (although it would be nice of course for the AMR
>         logic to
>         >         detect that the mask is nan and issue an error
>         message). The
>         >         origin is in the incomplete initialization of the
>         mask.
>         >
>         >         To convince myself of this, I switched poisoning
>         off, and
>         >         observed no odd behavior. I then switched it back on
>         and ran a
>         >         number of grid configurations where the mask was set
>         pointwise
>         >         (say, to coincide with one of the evolution
>         variables), and
>         >         again I had no trouble. As for the actual mask
>         (based on the
>         >         derivative of a grid function), I played a long time
>         with
>         >         scheduling the filling of interior points and
>         boundaries, but
>         >         always observed the issue that Hal is reporting.
>         Ideally, I'd
>         >         think that filling the interior, syncing, and
>         finally applying
>         >         outer/symmetry boundary conditions would work, but
>         that
>         >         doesn't seem to be the case.
>         >
>         >         Erik: when you suggest to set the boundary
>         explicitly, do you
>         >         mean the outer boundary? Since in this case we only
>         have a
>         >         symmetry boundary (periodic), do you mean we should
>         populate
>         >         that part of the grid independently of the symmetry
>         thorn?
>         >
>         >         Thanks,
>         >         Eloisa
>         >
>         >
>         >
>         >
>         > --
>         > Erik Schnetter <schnetter at cct.lsu.edu>
>         > http://www.cct.lsu.edu/~eschnett/
>         >
>         >
>         
>         
>         --
>         Hal Finkel
>         Postdoctoral Appointee
>         Leadership Computing Facility
>         Argonne National Laboratory
>         
>         
> 
> 
> 
> 
> -- 
> Erik Schnetter <schnetter at cct.lsu.edu>
> http://www.cct.lsu.edu/~eschnett/
> 
> 

-- 
Hal Finkel
Postdoctoral Appointee
Leadership Computing Facility
Argonne National Laboratory



More information about the Users mailing list