[ET Trac] [Einstein Toolkit] #488: Reduce space taken by Formaline tarballs

[Einstein Toolkit]

#488: Reduce space taken by Formaline tarballs
--------------------------+-------------------------------------------------
  Reporter:  hinder       |       Owner:  hinder   
      Type:  enhancement  |      Status:  accepted 
  Priority:  minor        |   Milestone:           
 Component:  Cactus       |     Version:           
Resolution:               |    Keywords:  Formaline
--------------------------+-------------------------------------------------

Comment (by eschnett):

 Ad 1: I think Simfactory should use a criterion that is not "Formaline".
 For example, Simfactory could treat all files larger than 1 MByte this
 way, or all archives, etc.

 This should not happen in Formaline, because it is much more difficult to
 do this in C than in Perl, and much more difficult to do this reliably at
 run time on an HPC system than after the fact. Also, doing this in
 Simfactory means that one can save space (again) after a simulation has
 been moved to a different system.

 Ad 2: Using Simfactory's cache directory seems like a good idea. However,
 there is currently no mechanism yet to "clean the cache" periodically,
 i.e. to remove old files that are present only in the cache. Also, the
 cache can hold only a single file for each name (but this may be good
 enough for Formaline tarballs).

 Note also that git already does all of these things (compress, de-
 duplicate). Maybe we should store the tarballs in git, or drop the
 tarballs in favour of having the source code in git. Formaline would still
 output them, but once Simfactory verified that the source tree is safely
 stored in git, it could delete the tarballs.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/488#comment:14>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit