#1584: CarpetIOHDF5 should support extensible datasets -------------------------+-------------------------------------------------- Reporter: hinder | Owner: eschnett Type: enhancement | Status: new Priority: minor | Milestone: Component: Carpet | Version: development version Keywords: | -------------------------+-------------------------------------------------- Currently, CarpetIOHDF5 outputs one dataset per iteration. This makes sense for gridfunctions, as the datasets are typically large and can change shape if there is regridding. However, for grid arrays, the size is fixed from one iteration to the next, and there are often only a few values of interest. This means that output consists of hundreds of separate datasets, all of the same shape, and each with only a handful of elements.
HDF5 supports "extensible datasets" http://www.hdfgroup.org/HDF5/Tutor/extend.html. These are datasets which can be extended with new data in some directions after being first created. I propose that CarpetIOHDF5 should support output of grid arrays using this format. Multipole already does this. We would add a dimension for "iteration" and extend the dataset in that dimension when new output is available. This would mean that a grid array could be read from a single dataset, rather than having to loop over all iterations.