[Users] ET test failures on Stampede

[Ian Hinder]

On 2 Nov 2014, at 17:07, Ian Hinder <ian.hinder at aei.mpg.de> wrote:

> 
> On 30 Oct 2014, at 20:41, Ian Hinder <ian.hinder at aei.mpg.de> wrote:
> 
>> 
>> On 30 Oct 2014, at 19:32, Steven R. Brandt <sbrandt at cct.lsu.edu> wrote:
>> 
>>> On 10/30/2014 01:30 PM, Erik Schnetter wrote:
>>>> On Thu, Oct 30, 2014 at 2:26 PM, Steven R. Brandt <sbrandt at cct.lsu.edu> wrote:
>>>>> On 10/30/2014 01:07 PM, Erik Schnetter wrote:
>>>>>> Ian
>>>>>> 
>>>>>> As a side remark, the executables created on Trestles are not
>>>>>> runnable. The test cases that nevertheless succeed on Trestles (!) are
>>>>>> probably not interesting.
>>>>> How does the test succeed if the executable can't run?
>>>> That's the question.
>>>> 
>>>> Cactus doesn't test whether the executable runs, it tests whether
>>>> every generated output file is correct. Maybe there are zero output
>>>> files.
>>> I thought it also checked exit code.
>> 
>> mpirun might not be propagating the exit code on that machine.
> 
> I think it's worse than that.  In WarnLevel.c, in CCTK_VWarn (called by CCTK_Warn), it says
> 
>  if (level <= error_level)
>  {
>    CCTK_Abort (NULL, 0);
>  }
> 
> The second argument to CCTK_Abort is the exit code of the process.  So if there is an "error" warning, the process exits with 0 exit code; i.e. success!  This happens in several places in this file.
> 
> The user guide does not say anything about the exit code of Cactus.  I think that if Cactus has a level-0 warning, i.e. an error, then it should exit with a non-zero exit code.  Is there a reason to exit "success" in this case?

Further, the test system seems to ignore the fact that Cactus exits with a nonzero exit code.  It displays

Cactus exited with error code 1
Please check the logfile...

  No files created in test directory

  Success: 0 files identical

And in the summary at the end, treats this as a passing test.  In this case, there were no test reference files and no files output, because the test does not produce any data, it just aborts if the test fails.

-- 
Ian Hinder
http://numrel.aei.mpg.de/people/hinder