If it is possible to generate the index files as post-processing step, then I would prefer that, as it reduces the time the simulation spends waiting for I/O. We could add it e.g. to Simfactory's cleanup step to have it happen automatically.
It is possible; the index files are identical to the original files, but with no data written to the datasets. People don't visualise their data enough, and having the index files makes it more likely that people will visualise their data, as reading the data will be faster. Having an additional step of post-processing the data is annoying.
How well-documented are the index files? If they are generated by default, then there should be a section in CarpetIOHDF5's thorn guide (or in the Visit reader's documentation) describing why they are a good idea, and what information they contain, and how they can be used to speed up input.
I don't think there is documentation for them apart from the description of the parameter in param.ccl.
It sounds like you don't want the additional index files appearing in users' output directories without them understanding what they are. I agree that having the index files in the output directory is a bit ugly. Another alternative would be to embed the content of the index files in the original HDF5 files when the simulation terminates, or periodically. The index files are small, so this should not be a large overhead. This would be a binary dataset which could be accessed by name without iterating all the datasets in the file.
_______________________________________________
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users