Hi.
I'm running a vacuum BHB evolution with a larger-than-usual set of inner refinement regions (levels 8, 9, 10 11 have radii of 12M, 8M, 6M, and 4M, respectively) and consequently the memory usage is a bit higher than normal. But I'm finding that it jumps up almost 100% after the first regridding, and stays there.
My diagnostic for this is the result of top on each of the nodes (via "qtop.pl", a script on the machine I'm using). Sampled before the first regridding, it shows each core using ~ 1.5% of the node's total memory, while after regridding, it's more like 2.9% (these are Sandy Bridge nodes, with 16 available cores).
However, the periodic output message from Carpet reporting the Grid structure etc. shows regions only marginally larger than before, and --- crucially for me --- has a marginally larger "Total required memory" (164 GB -> 167 GB, for instance).
So (a) what's using the extra memory, and (b) why isn't Carpet reporting it? How seriously should I be taking that "Total required memory" message?
I see this with executables generated from both the last (ET_2012_11) and current (ET_2013_05) stable releases, BTW. I'm attaching the current parameter file and SCROUT from a run.
Thanks,
Bernard