On Thu, Nov 19, 2015 at 11:05:24PM -0600, Frank Loeffler wrote:
The best way would probably be to count them locally on each process, write the result in a distributed grid array, and do the sum there by syncing the array and counting locally (not sure if a reduction on such an array would work, it might).
One complication here would be that, with AMR at least, you would need to take care of masking non-evolved points (ghost, boundary, coarse grid points inside fine grids, ...).
Another, but more expensive way (in terms of memory size and memory accesses) would be to create another grid function, populate it with either 1 or 0 depending on the non-zero-property of the GF you are interested in, and calling the existing sum-reduction on it.
Here you wouldn't need to care about masking, as the Carpet reduction would do that for you.
Frank