If it is possible to generate the index files as post-processing step, then I would prefer that, as it reduces the time the simulation spends waiting for I/O. We could add it e.g. to Simfactory's cleanup step to have it happen automatically.
How well-documented are the index files? If they are generated by default, then there should be a section in CarpetIOHDF5's thorn guide (or in the Visit reader's documentation) describing why they are a good idea, and what information they contain, and how they can be used to speed up input.
-erik
On Wed, Feb 27, 2013 at 2:41 PM, Roland Haas <roland.haas@physics.gatech.edu
wrote:
Hello all,
Would somebody object to making CarpetIOHDF5::output_index = "yes" the default? (It is currently "no".)
It should not hurt to create these small files. (Or, does it?)
It creates files which in itself might be bad. Creating files can be an issue on some file systems where creating files is slow (or there might be limit on the number of files, that apparently on some clusters [admittedly I know of no Cactus users there] can be as low as 1e6 files *total*).
Also currently there is a bug/mis-feature in that output_index writes indices for both data and checkpoint files. The later probably should have a separate switch since index files are less useful there.
If the issue is that we currently have no method for users to create index files for existing HDF5 files once eg they find that visualization is slow, then for that I have a modified version of hdf5_merge (or was it hdf5_extract... anyway one of them) where I simply commented out the final H5Dwrite and which generates perfectly fine index files (of the "new" format that do not have the extra attribute with just that change or "old" ones with the attribute with the obvious changes [that are also in]). It's of course a bit of a hack since the tools were not originally meant to do that.
Yours, Roland
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net. _______________________________________________ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users