Frank,

On Feb 21, 2013, at 8:05 PM, Frank Loeffler wrote:

On Thu, Feb 21, 2013 at 06:31:23PM -0700, Bruno Giacomazzo wrote:
I found out that the problem is due to 1D ASCII output
(which is active in that parfile) and that makes the runs
particularly slow above ~100 cores on this machine.

That is not not just on this machine. xD output with x>=1 doesn't scale.
Anywhere. Use HDF5, except for debugging where ASCII might be more
convenient and you don't care about performance. There isn't much you
could do about that without changing CarpetIOASCII.

thank you for your comment. It's the first time that I see such a strong impact on runs using only ~100 cores. One of the reasons I did these tests was also that I was experiencing some problems with one of my production runs that was slower on Stampede than on Ranger.

and hence I didn't plot them in the other figure). I didn't try
using 1D hdf5 output instead, does anyone use it?

Not 1D, but 2D, and that works very well.

I use 2D hdf5, but sometimes it's useful to have also 1D data (without having to postprocess 2D or 3D hdf5 files). I never used the 1D hdf5 output available in Carpet and I was wondering if other people use it.

Cheers,
Bruno

Dr. Bruno Giacomazzo
JILA - University of Colorado
440 UCB
Boulder, CO 80309
USA

Tel.  : +1 303-492-5170
Fax  : +1 303-492-5235

----------------------------------------------------------------------
There are only 10 types of people in the world:
Those who understand binary, and those who don't
----------------------------------------------------------------------