[ET Trac] [Einstein Toolkit] #1568: Reduce overhead of Formaline

[Einstein Toolkit]

#1568: Reduce overhead of Formaline
-------------------------+--------------------------------------------------
 Reporter:  hinder       |       Owner:                     
     Type:  enhancement  |      Status:  new                
 Priority:  major        |   Milestone:                     
Component:  Cactus       |     Version:  development version
 Keywords:               |  
-------------------------+--------------------------------------------------
 The Formaline thorn, used for storing important information about a
 simulation in the simulation output, currently has some performance and
 size overheads which discourage people from using it.  These should be
 reduced or mitigated to encourage people to use this important thorn.

 This ticket is based on discussion in #1565.

 Problems with Formaline:
   * During compilation, Formaline stores the built source tree in a
 cactusjar.git repository. My impression is that this takes a lot of time
 on slow filesystems.
   * During compilation, Formaline stores the whole source tree as tarballs
 in the executable.  My impression is that this is also slow.
   * The source tarballs output in the simulation output directory are
 often many times larger than the rest of the simulation, and cause a lot
 of overhead when transferring simulations for analysis.  This might be
 improved by a better sync tool (simfactory 3?) which identified that
 tarballs were identical to those which have already been transferred, and
 skipped the transfer.  I would like that the source tarballs are identical
 if the contents are identical, even if Cactus has been rebuilt.  I think I
 observed that this was not the case (maybe a build ID is included?).
   * As an aside: there is a message at link time that Formaline has
 finished doing something, but this is misleading; Formaline has other
 tasks which it does after this message is displayed; the wait for the
 final executable is not due solely to linking, but also to some archiving
 task of formaline.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1568>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit