#764: cached output in CarpetIOSCalar --------------------------+------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: enhancement | Status: review Priority: minor | Milestone: Component: Carpet | Version: Resolution: | Keywords: --------------------------+-------------------------------------------------
Comment (by rhaas):
I really really detest this Spam filter. This is the second time I had to type my comments since the spam filter happily removed my posting and my browser (Opera) did not restore the text in the box.
Using pointers instead of copying the cached_stream objects seems like a good idea.
The number of file limit can be reached if we request scalar output for the majority of our grid functions. I have seen limits as low as 1024 files per process (reachable with hdf5_slicer and HDF5 3D output from 576 MPI processes).
I order to avoid loss of data when the simulation fails any caching should flush its caches during checkpointing (I believe ther eis a bin for it). This way all currently cached data will be reproduced when the user recovers from the last checkpoint.
I can easily change CarpetIOScalar's code (trunk version) to have different columns for the individual reductions. This is very straigthforward with the exception of the case where the requested reductions change during the run (where I would opt to output a second header with the column numbers). Note that this logic will fail if the user changes the requested reductions when recovering from a checkpoint (unless I track the parameter value out of the checkpoint and what is seen the next time the code is called, not sure though if the flesh lets me see the parameter values from the checkpoint).