[ET Trac] #2543: Consolidate data formats to simplify postprocessing

[Wolfgang Kastaun]

#2543: Consolidate data formats to simplify postprocessing

 Reporter: Wolfgang Kastaun
   Status: new
Milestone: 
  Version: development version
     Type: enhancement
 Priority: minor
Component: 

Comment (by Wolfgang Kastaun):

The proposed solution for CarpetX would probably simplify the logic which gathers information on the available data a lot. I’m not entirely sure about speed. The reason parsing the hdf5 files takes so long seems a design flaw that requires basically to read the whole file just to get the names of all datasets, combined with the unfortunate choice of having the table of content information only in those names. I also did not get how the solution would look like for runs using many nodes, would there be one file for each node or even MPI process, or is the information collected first on one node? 

But in principle, reading a few thousand yaml files should not take that long, we would have to try.

--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2543/consolidate-data-formats-to-simplify
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/trac/attachments/20210803/8dad45b6/attachment.html