[ET Trac] [Einstein Toolkit] #2113: Test system ignores exit code of Cactus

[Einstein Toolkit]

#2113: Test system ignores exit code of Cactus
--------------------+-------------------------------------------------------
 Reporter:  hinder  |       Owner:                     
     Type:  defect  |      Status:  new                
 Priority:  major   |   Milestone:                     
Component:  Cactus  |     Version:  development version
 Keywords:          |  
--------------------+-------------------------------------------------------
 When a test is running, and Cactus fails with a nonzero exit code, the
 test system populates a data structure to indicate that the test has
 failed:
 {{{
 RunTestUtils.pl:1618, RunTest():
     $testdata->{"$thorn FAILED"} .= "$parfile ";
     $testdata->{"NFAILED"}++;
 }}}
 but the test is later regarded as successful.  Probably the function
 CompareTestFiles ignores the fail code set by RunTest.  In the case where
 the test has data file to compare with, this usually is exhibited as a
 failure, as the new files will probably not be created.  But in the case
 where there are no data files to compare with, the system doesn't expect
 to find any, and the test succeeds.  CompareTestFiles probably needs to be
 modified to take into account the failed run, or maybe it should never be
 called in the first place.

 The test Carpet/Carpet/test/64k2.par currently shows this problem; it
 reports that it is unable to allocate 12 GB of RAM, and exits like this:

 {{{
 ------------------------------------------------------------------------

   Test Carpet: 64k2
     "Carpet/Carpet/test/64k2.par"

   Issuing rm -f output-0000-active && ln -s . output-0000-active && mkdir
 -p SIMFACTORY &&
 TESTSUITE_PARFILE=/home/jenkins/workspace/simulations/EinsteinToolkit_f297f2e499015361358e5acfe294e024673292e3_1/output-0000/arrangements/Carpet/Carpet/test/64k2.par
 /home/jenkins/workspace/EinsteinToolkit/../simulations/EinsteinToolkit_f297f2e499015361358e5acfe294e024673292e3_1/output-0000/SIMFACTORY/RunScript
 Cactus exited with error code 134
 Please check the logfile
 /home/jenkins/workspace/simulations/EinsteinToolkit_f297f2e499015361358e5acfe294e024673292e3_1/output-0000/TEST/sim/Carpet/64k2.log


   No files created in test directory

   Success: 0 files identical

 ------------------------------------------------------------------------
 }}}

 The test is then considered as successful in the final output.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/2113>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit