[Users] test suite data files commits to svn

[Ian Hinder]

On 19 Apr 2010, at 16:33, Erik Schnetter wrote:

> On Apr 19, 2010, at 2:43 , Ian Hinder wrote:
>
>>
>> On 19 Apr 2010, at 05:21, Baiotti Luca wrote:
>>
>>> Hi Bruno,
>>>
>>> the idea of the testsuites is to have reliable data (corresponding
>>> to a given
>>> parfile) to compare with, when one wants to check that the code
>>> works , e.g., on a
>>> new machine or that a change in the source code has not introduced
>>> unwanted
>>> behaviour. So the data are a fundamental part of the testsuites.
>>
>> While I completely agree with what Luca said, I would like to echo
>> Bruno's concern that some of the test suite data is very large. Since
>> it is only designed for regression testing, there is no reason I can
>> see to have such large data files.  There is either a regression or
>> there isn't, and once you have found that it is present, you can then
>> do more detailed runs with more output to locate it.  You shouldn't
>> need a large amount of output.
>
>
> If a thorn calculates N quantities, then ideally there should be N  
> 3D output files, so that all quantities are tested.  It is  
> impossible to test correctness if there are only norms, and if there  
> is only 1D output then problems off-axis are not detected.

I agree about norms and 1D being insufficient, but effort should be  
made to ensure that these files are small.  For example, they don't  
need to contain a very large number of points.

> In a way, the best approach would be to create a checkpoint file  
> from a test parameter file and store this.  This would be the most  
> efficient way to store the complete information.  Unfortunately,  
> Cactus test cases support only ASCII output (so far?).

I think this would store much too much information.  Yes, it would be  
good if the test suite mechanism supported hdf5 files.  Anyone  
interested in adding this might look into the h5diff tool...

-- 
Ian Hinder
ian.hinder at aei.mpg.de