If it is possible to generate the index files as post-processing step, then I would prefer that, as it reduces the time the simulation spends waiting for I/O. We could add it e.g. to Simfactory's cleanup step to have it happen automatically.

How well-documented are the index files? If they are generated by default, then there should be a section in CarpetIOHDF5's thorn guide (or in the Visit reader's documentation) describing why they are a good idea, and what information they contain, and how they can be used to speed up input.

-erik



On Wed, Feb 27, 2013 at 2:41 PM, Roland Haas <roland.haas@physics.gatech.edu> wrote:
Hello all,

> Would somebody object to making CarpetIOHDF5::output_index = "yes" the
> default? (It is currently "no".)
>
> It should not hurt to create these small files. (Or, does it?)
It creates files which in itself might be bad. Creating files can be an
issue on some file systems where creating files is slow (or there might
be limit on the number of files, that apparently on some clusters
[admittedly I know of no Cactus users there] can be as low as 1e6 files
*total*).

Also currently there is a bug/mis-feature in that output_index writes
indices for both data and checkpoint files. The later probably should
have a separate switch since index files are less useful there.

If the issue is that we currently have no method for users to create
index files for existing HDF5 files once eg they find that visualization
is slow, then for that I have a modified version of hdf5_merge (or was
it hdf5_extract... anyway one of them) where I simply commented out the
final H5Dwrite and which generates perfectly fine index files (of the
"new" format that do not have the extra attribute with just that change
or "old" ones with the attribute with the obvious changes [that are also
in]). It's of course a bit of a hack since the tools were not originally
meant to do that.

Yours,
Roland

--
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
_______________________________________________
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users



--
Erik Schnetter <schnetter@cct.lsu.edu>
http://www.perimeterinstitute.ca/personal/eschnetter/