Re: [Users] [Einstein Toolkit] #66: IOHDF5::out3D_ghosts and friends doesn't work and corrupts 2D data slices
On 18 Oct 2011, at 15:51, Einstein Toolkit wrote:
#66: IOHDF5::out3D_ghosts and friends doesn't work and corrupts 2D data slices -----------------------------------+---------------------------------------- Reporter: bcmsma@… | Owner: eschnett Type: defect | Status: new Priority: minor | Milestone: Component: Carpet | Version: Resolution: | Keywords: -----------------------------------+----------------------------------------
Comment (by rhaas):
The output_XXX ones work for out3d_vars (the "sliced" 3d output), they do not work for the legacy out_vars 3d output as you correclty point out. The former should produce identical output for grid functions (and maybe scalar and arrays though that is much less well tested). Its code cannot be used for checkpointing though which is why the old legacy code is still in (and I don't fully trust the new 3D code yet since it has not been active for very long).
I wasn't really aware of the new-style 3D output method. Does this method allow for parallel output (i.e. one file per process)? If not, then I worry that it might be quite slow.
Hello Ian, all,
I wasn't really aware of the new-style 3D output method. Does this method allow for parallel output (i.e. one file per process)? If not, then I worry that it might be quite slow.
It does by respecting IOUtil::out_mode and IOUtils::out_proc_every. Note though that the output files have different names: they contain a .xyz. fragment the same way that 3d ASCII data does. By default the IOUtil parameters are such that one output file per processor is created (ie. the old behaviour). 1d and 2d HDF5 output ignores the IOUtil parameters to remain backwards compatible (and because it does not make much sense to spread a couple 100 kB of output data across all processors). You could even set it up so that it writes only on say 8 out of 512 MPI processes possibly not overwhelming the cluster's IO subsystem and making us much more well liked by the admins :-) It might even be faster since the number of file open/file close calls goes down (which seem to be slow on lustre file systems).
Yours, Roland
users@lists.einsteintoolkit.org
-
Ian Hinder -
Roland Haas