Hello, I am using the Einstein Toolkit for cosmological simulations, and would like to ask a question regarding my HDF5 3D output. Currently I am outputting all 3D snapshots to one HDF5 file (as per default). I would like to have my output as separate files for each time snapshot, to help with ease of copying and visualisation in post processing.
I found the following parameter in IOUtil, and changed it accordingly for a simulation: IO::out_timesteps_per_file = 1 However this made no difference to my output, I still had multiple time snapshots in the one file.
Is there another way to separate my 3D HDF5 output into one file per snapshot? Any help anyone can offer would be much appreciated!
Kind regards, Hayley Macpherson
Hayley
The simplest way would be to post-process the HDF5 files after the simulation has finished. HDF5 comes with tools that manipulate HDF5 files, e.g. adding/removing compression, and also copying or selecting objects from one file to another. You can also use this to combine output from all processes into a single file. For post-processing, having one file per variable (!) and iteration is often the most convenient.
-erik
On Tue, Jun 20, 2017 at 9:04 AM, Hayley Macpherson < hayley.macpherson@monash.edu> wrote:
Hello, I am using the Einstein Toolkit for cosmological simulations, and would like to ask a question regarding my HDF5 3D output. Currently I am outputting all 3D snapshots to one HDF5 file (as per default). I would like to have my output as separate files for each time snapshot, to help with ease of copying and visualisation in post processing.
I found the following parameter in IOUtil, and changed it accordingly for a simulation: IO::out_timesteps_per_file = 1 However this made no difference to my output, I still had multiple time snapshots in the one file.
Is there another way to separate my 3D HDF5 output into one file per snapshot? Any help anyone can offer would be much appreciated!
Kind regards, Hayley Macpherson
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Hi Erik, Ian, Roland
Thanks so much for your replies!
Erik - this sounds like the easiest way. I did notice these utilities, however I can't find documentation on how to use them. Am I missing it somewhere?
Ian - yes, I did see that loop. I added a print statement inside it, and it looks like the code isn't even going into that loop (with out_timesteps_per_file=1 of course). I don't see any files with the "iter" suffix, and I am definitely looking at a new simulation directory. I'll have a more in depth look at this in the coming weeks. Thanks!
Roland - thanks for your suggestion. I'll give this a go if the post-processing method doesn't work for me.
Again, thanks all for you help. Cheers Hayley
On 23 June 2017 at 01:55, Erik Schnetter schnetter@cct.lsu.edu wrote:
Hayley
The simplest way would be to post-process the HDF5 files after the simulation has finished. HDF5 comes with tools that manipulate HDF5 files, e.g. adding/removing compression, and also copying or selecting objects from one file to another. You can also use this to combine output from all processes into a single file. For post-processing, having one file per variable (!) and iteration is often the most convenient.
-erik
On Tue, Jun 20, 2017 at 9:04 AM, Hayley Macpherson < hayley.macpherson@monash.edu> wrote:
Hello, I am using the Einstein Toolkit for cosmological simulations, and would like to ask a question regarding my HDF5 3D output. Currently I am outputting all 3D snapshots to one HDF5 file (as per default). I would like to have my output as separate files for each time snapshot, to help with ease of copying and visualisation in post processing.
I found the following parameter in IOUtil, and changed it accordingly for a simulation: IO::out_timesteps_per_file = 1 However this made no difference to my output, I still had multiple time snapshots in the one file.
Is there another way to separate my 3D HDF5 output into one file per snapshot? Any help anyone can offer would be much appreciated!
Kind regards, Hayley Macpherson
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
-- Erik Schnetter schnetter@cct.lsu.edu http://www.perimeterinstitute.ca/personal/eschnetter/
On 20 Jun 2017, at 15:04, Hayley Macpherson hayley.macpherson@monash.edu wrote:
Hello, I am using the Einstein Toolkit for cosmological simulations, and would like to ask a question regarding my HDF5 3D output. Currently I am outputting all 3D snapshots to one HDF5 file (as per default). I would like to have my output as separate files for each time snapshot, to help with ease of copying and visualisation in post processing.
I found the following parameter in IOUtil, and changed it accordingly for a simulation: IO::out_timesteps_per_file = 1 However this made no difference to my output, I still had multiple time snapshots in the one file.
Is there another way to separate my 3D HDF5 output into one file per snapshot? Any help anyone can offer would be much appreciated!
Hi Hayley,
Looking at the code, I don't immediately see why it's not working. If you look in the file
arrangements/Carpet/CarpetIOHDF5/src/CarpetIOHDF5.cc
in the function OutputVarAs, around line 895, there is the code for outputting a fixed number of timesteps in a given file.
if (out_timesteps_per_file > 0) { // Round down to nearest multiple of out_timesteps_per_file int const iter = cctk_iteration / out_timesteps_per_file * out_timesteps_per_file; char buffer[32]; snprintf(buffer, sizeof(buffer), ".iter_%d", iter); filename.append(buffer); }
Maybe you can add some debugging code to work out whether this code is being executed or not, and why it's not doing the right thing.
Do you get a new file with the ".iter_..." suffix? Are you perhaps looking at an old file without this suffix which was left over from earlier runs?
Hello Hayley,
Is there another way to separate my 3D HDF5 output into one file per snapshot?
One file per snapshot with output from all processors is likely not (easily) possible. Similarly there is currently no code in CarpetIOHDF5 for creating a new file every N output cycles.
Any help anyone can offer would be much appreciated!
You'd have to edit
repos/carpet/CarpetIOHDF5/src/OutputSlice.cc
(for out3d_vars output). Simplest would be to edit the basefilename that you can see in line 441:
ostringstream basefilenamebuf; basefilenamebuf << my_out_slice_dir << "/" << alias; const string basefilename = basefilenamebuf.str();
as long as you choose a new filename, everything should be fine. Something like (untested!):
basefilenamebuf << ".it_" << cctk_iteration
should do the trick (one file per iteration and with the obvious changes for multiple iterations per file).
Yours, Roland
users@lists.einsteintoolkit.org
-
Erik Schnetter -
Hayley Macpherson -
Ian Hinder -
Roland Haas