Hi Erik.

I was originally using out3D_vars etc, but the resulting output was always *chunked* and *cumulative* over time. I don't see a parameter that controls this behavior in CarpetIOHDF5/param.ccl.

Bernard

On 6 January 2016 at 14:51, Erik Schnetter schnetter@cct.lsu.edu wrote:

CarpetIOHDF5 has two output modes. You can either set out_vars, giving the traditional chunked output, or you can set out3D_vars, which uses a different output mechanism and slightly different file names. The latter may be unchunked by default. (I hope.)

(There is also out1D_vars and out2D_vars.)

-erik

On Wed, Jan 6, 2016 at 2:39 PM, Bernard Kelly physicsbeany@gmail.com wrote:

...

Hi. Silly question time:

I'd like to write some grid functions to disk as HDF5 3D files that are *unchunked*, but one per output time step. This I'm doing by setting the following parameters:

IOUtil::out_mode = "onefile" IOUtil::out_timesteps_per_file = 1

However, an unexpected (to me) side-effect of this is that my checkpoint files are now also unchunked, which leads to longer recovery (and occasionally failure).

So ... can I have chunked checkpoint files, but unchunked everything else 3D using CarpetIOHDF5?

Thanks,

Bernard _______________________________________________ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users

-- Erik Schnetter schnetter@cct.lsu.edu http://www.perimeterinstitute.ca/personal/eschnetter/

Hi Roland.

Thanks for the suggestion and code snippets. I tried them out on a small test case with two variants, and they seemed to work OK:

variant 1: I made a new parameter "unchunked3D" so that I could turn this on and off:

if ((outdim == 3) && unchunked3D) { // don't split files nioprocs = 1; ioproc = 0; ioproc_every = dist::size(); } else if (outdim == 3) { // only 3D output splits files const ioGH *IO = (ioGH *) CCTK_GHExtension (cctkGH, "IO"); assert(IO); nioprocs = IO->nioprocs; } else {...

variant 2: To solve the single-file-per-timestep issue, I needed "cctkGH->cctk_iteration", rather than just "cctk_iteration"

basefilenamebuf << my_out_slice_dir << "/" << alias << ".it_" << cctkGH->cctk_iteration;

The problem is, now my 2D HDF5 output is *also* split by time step, which I didn't want/need.

Bernard

On 6 January 2016 at 16:24, Roland Haas rhaas@aei.mpg.de wrote:

Hello Bernard,

...

I was originally using out3D_vars etc, but the resulting output was always *chunked* and *cumulative* over time. I don't see a parameter that controls this behavior in CarpetIOHDF5/param.ccl.

Hmm, in that case you will have to modify C++ code. In CarpetIOHDF5's OutputSlice.cc near the end of the routine CheckSteerableParameters (around line 215 in my checkout) you will see code: --8<-- // copy ioprocs and ioproc ot if (outdim == 3) { // only 3D output splits files const ioGH *IO = (ioGH *) CCTK_GHExtension (cctkGH, "IO"); assert(IO); nioprocs = IO->nioprocs; ioproc = IO->ioproc; ioproc_every = IO->ioproc_every; //cout << "nioprocs: " << nioprocs << " ioproc: " << ioproc << endl; } else { nioprocs = 1; ioproc = 0; ioproc_every = dist::size(); } --8<-- which tells you that out1d and out2d are always unchunked (one file) and out3d respects nioprocs which is what "onefile" sets. So in order to get unchuncked 3d output you'd have to make the outdim == 3 case look like the else case. Note that the current code does what it ought to do: it respects your "onefile" choice.

The single file issue is a bug, and needs a fix. The easiest by far will be to add ".it_XXX" to basefilenamebuf in OutputVarAs around line 461, ie make it look like this:

--8<-- ostringstream basefilenamebuf; basefilenamebuf << my_out_slice_dir << "/" << alias << ".it_" << cctk_iteration; const string basefilename = basefilenamebuf.str(); --8<--

mind you, this is a hack though :-) it really should be controlled by the out_timesteps_per_file option.

Yours, Roland

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

---

Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users