#488: Reduce space taken by Formaline tarballs --------------------------+------------------------------------------------- Reporter: hinder | Owner: hinder Type: enhancement | Status: accepted Priority: minor | Milestone: Component: Cactus | Version: Resolution: | Keywords: Formaline --------------------------+-------------------------------------------------
Comment (by eschnett):
Ad 1: I think Simfactory should use a criterion that is not "Formaline". For example, Simfactory could treat all files larger than 1 MByte this way, or all archives, etc.
This should not happen in Formaline, because it is much more difficult to do this in C than in Perl, and much more difficult to do this reliably at run time on an HPC system than after the fact. Also, doing this in Simfactory means that one can save space (again) after a simulation has been moved to a different system.
Ad 2: Using Simfactory's cache directory seems like a good idea. However, there is currently no mechanism yet to "clean the cache" periodically, i.e. to remove old files that are present only in the cache. Also, the cache can hold only a single file for each name (but this may be good enough for Formaline tarballs).
Note also that git already does all of these things (compress, de- duplicate). Maybe we should store the tarballs in git, or drop the tarballs in favour of having the source code in git. Formaline would still output them, but once Simfactory verified that the source tree is safely stored in git, it could delete the tarballs.