[Users] Problem with CarpetRegrid2/AMR

[Hal Finkel]

On Tue, 2011-08-30 at 21:06 -0400, Erik Schnetter wrote:
> On Tue, Aug 30, 2011 at 5:28 PM, Hal Finkel <hfinkel at anl.gov> wrote:
> > Could I also decrease the block size? I currently have
> > CarpetRegrid2::adaptive_block_size = 4, could it be smaller than that?
> > Is there a restriction based on the number of ghost points?
> 
> Yes, you can reduce the block size. I assume that both the regridding
> operation and the time evolution will become slower if you do that,
> because more blocks will have to be handled.

Regardless of what I do, once we get past the first coarse time step,
the program seems to "hang" at "INFO (Carpet): [ml=0][rl=0][m=0][tl=0]
Regridding map 0...".

Overall, it is in dh::regrid(do_init=true). It spends most of its time
in bboxset<int, 3>::normalize() and, specifically, mostly in the loop:
for (typename bset::iterator nsi = nbs.begin(); nsi != nbs.end(); ++
nsi). The normalize() function does exit, however, so it is not hanging
in that function.

The core problem seems to be that it takes a long time to execute:
boxes  = boxes .shift(-dir) - boxes;
in dh::regrid(do_init=true). Probably because boxes has 129064 elements.
The coarse grid is now only 30^3 and I've left the regrid box size at 4.
I'd think, then, that the coarse grid should have a maximum of 30^3/4^3
~ 420 refinement regions.

What is the best way to figure out what is going on?

Thanks again,
Hal

> This will also require
> more memory, since each block carries its own ghost zone overhead.
> 
> No, there is no such restriction.
> 
> -erik
> 

-- 
Hal Finkel
Postdoctoral Appointee
Leadership Computing Facility
Argonne National Laboratory