[ET Trac] [Einstein Toolkit] #1283: Missing data in HDF5 files

[Einstein Toolkit]

#1283: Missing data in HDF5 files
---------------------+------------------------------------------------------
  Reporter:  hinder  |       Owner:     
      Type:  defect  |      Status:  new
  Priority:  major   |   Milestone:     
 Component:  Cactus  |     Version:     
Resolution:          |    Keywords:     
---------------------+------------------------------------------------------

Comment (by eschnett):

 I would not use "tmp" to indicate files are currently incomplete. What
 about "incomplete"? Or "$$$"?

 One either needs to store the data belonging to a simulation, or re-run
 the simulation if there is a problem. If archives can't handle checkpoint
 files, and if local data disks aren't large enough to keep checkpoint
 files, then re-running is the correct response. People need to be aware of
 this, and in this mode, all but the last (verified to be safely usable)
 checkpoint files should be deleted, probably semi-automatically.

 If a data disk has 300 Tbyte, then I would expect an archive to have 3,000
 Tbyte. Alternatively, one could design a system that has 100 Tbyte data
 disk and 1,000 Tbyte usable archive space (10 Gbit interconnect?).
 Currently, since no one uses archive except to stash data they don't
 intend to look at any more, the archive is difficult to use, and there is
 a large pressure to increase data disks, further reducing the need to use
 an archive, and thus reducing pressure to make it easier to use.

 I meant checkpoint files. But, thinking about this, it could equally apply
 to restarts, or at least large output files in restarts.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1283#comment:12>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit