[Users] visualization-friendly HDF5 format

[Frank Loeffler]

Hi,

We had quite a discussion about the problems the current visualization
tools have reading our hdf5 format. It became clear that we have to
change/extend it, but before we think about how, I would like to see
what specifically the readers have problems with. Let me start here with
what I can recall, and please freely add to that.

One of the problems is that readers currently have to iterate over all
datasets present to get to some information they need. Getting the list
of all datasets by itself can take quite a while on a regular hdf5 file,
but then readers also have to look at attributed within each dataset.
While of of this is necessary to visualize all data in a given file,
most of the time not all of the data is actually necessary, and
certainly not for just 'opening the file'.

operations a reader needs to be fast:
- list of variables (at a given iteration)
- list of time steps / iterations
- AMR structure for one given iteration (all maps, rls and components)

Regardless of these additional meta-data, we already established the
need for a meta-data file, effectively a copy of all datasets but
without the actual data. Am I remembering correctly that the idea was to
write this at run-time (eventually - right now it could be generated as
post-processing)?

Is this all readers would need?

Frank

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20150818/9815fe5b/attachment.bin