#764: cached output in CarpetIOSCalar --------------------------+------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: enhancement | Status: review Priority: minor | Milestone: Component: Carpet | Version: Resolution: | Keywords: --------------------------+-------------------------------------------------
Comment (by rhaas):
I also request that the spam filter be removed. I updated the original caching patch to not use reference counting anymore (much nicer this way) and to also flush the caches in the CHECKPOINT bin to ensure that no data is lost when jobs are not cleanly terminated. I attach a second patch to implement an option all_reductions_in_one file that implements Erik's suggestion. The basic idea is simple: open only a single file outside of the loop over all requested reductions instead of a new file for each reduction. However due to the state-caching, IO timing calls and the desire to have somewhat sensible headers, column arrangements, there are a large number of if (all_reductions_in_one_file) clauses that are used to effectively move code blocks from inside the loop over reductions to the outside. I provide a (new) test and have tested that the code passes the old test if the parameter is off. Please note that currently both patches are against a vanilla CarpetIOScalar code. They cannot be both applied (though merging them is simple).