[ET Trac] [Einstein Toolkit] #566: Support compressed (tar.gz) testsuite data

[Einstein Toolkit]

#566: Support compressed (tar.gz) testsuite data
-------------------------+--------------------------------------------------
 Reporter:  knarf        |       Owner:                     
     Type:  enhancement  |      Status:  new                
 Priority:  minor        |   Milestone:  ET_2011_11         
Component:  Cactus       |     Version:  development version
 Keywords:               |  
-------------------------+--------------------------------------------------
 This patch to Cactus adds support for compressed testsuite data.

 It assumes that if no directory is found within the thorn containing the
 reference data for a particular test, that it might be in a
 correspondingly named file (test.par with a missing test/ as output_dir
 might actually have test.tar.gz). Cactus would, when running the
 testsuites, decompress the tarfile to TEST/config/thorn/test.orig, and
 compare that against the new results (in TEST/config/thorn/test, as
 usual). The test.orig directory will remain within TEST for easier
 debugging/comparing later,
 but will be overwritten for each new re-run of the test.

 Using ET_2011_05 as example, compressing all testsuites this way, and
 disregarding the svn metadata, this reduces the number of files within
 arrangements (following symlinks) from 16k to 4.8k, and the size from
 220MB to 112MB. The metadata from svn is likely to about double these
 numbers. These numbers mean that using this would not only save space, but
 also a lot of time when syncing between machines.

 I propose to add support for this before the next ET release, but to only
 actually use it in thorns after this.

 Commands you may find useful when testing this:

 Compress all testsuite data (from within arrangements/):

 for i in `find -L -maxdepth 3 -mindepth 3 -type d -name
 test`; do cd $i; find -maxdepth 1 -mindepth 1 -not -name .svn -type d |
 awk '{print $1".tar.gz "$1;}' | xargs -n 2 -P 4 tar -czf; cd -; done

 After that: remove the test data directories (because they have higher
 priority than the tar.gz files):

 find -L -maxdepth 4 -mindepth 4 -type d | grep -v .svn | grep -e
 '\/test\/' | xargs rm -rf

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/566>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit