[Users] test suite data files commits to svn

[Erik Schnetter]

It would be good to have a "dump-all-variables" parameter in the I/O  
thorns.  This would simplify the following scenario:

The test cases would be set up to output only a small amount of  
information.  dump-all-variables is used to output all variables when  
the test case is created or updated, and these values are stored in a  
different repository.  If a difference is detected, dump-all-variables  
could be used to quickly find differences after downloading the  
additional data.

-erik

On Apr 20, 2010, at 7:11 , Allen Gabrielle wrote:

> Not sure if it is useful for detecting errors or for finding the  
> source of a problem once a problem is detected,  but we could keep  
> larger files or complete data sets remotely on a server.
>
> On Apr 20, 2010, at 4:35 AM, Ian Hinder wrote:
>
>>
>> On 19 Apr 2010, at 16:33, Erik Schnetter wrote:
>>
>>> On Apr 19, 2010, at 2:43 , Ian Hinder wrote:
>>>
>>>>
>>>> On 19 Apr 2010, at 05:21, Baiotti Luca wrote:
>>>>
>>>>> Hi Bruno,
>>>>>
>>>>> the idea of the testsuites is to have reliable data (corresponding
>>>>> to a given
>>>>> parfile) to compare with, when one wants to check that the code
>>>>> works , e.g., on a
>>>>> new machine or that a change in the source code has not introduced
>>>>> unwanted
>>>>> behaviour. So the data are a fundamental part of the testsuites.
>>>>
>>>> While I completely agree with what Luca said, I would like to echo
>>>> Bruno's concern that some of the test suite data is very large.  
>>>> Since
>>>> it is only designed for regression testing, there is no reason I  
>>>> can
>>>> see to have such large data files.  There is either a regression or
>>>> there isn't, and once you have found that it is present, you can  
>>>> then
>>>> do more detailed runs with more output to locate it.  You shouldn't
>>>> need a large amount of output.
>>>
>>>
>>> If a thorn calculates N quantities, then ideally there should be N
>>> 3D output files, so that all quantities are tested.  It is
>>> impossible to test correctness if there are only norms, and if there
>>> is only 1D output then problems off-axis are not detected.
>>
>> I agree about norms and 1D being insufficient, but effort should be
>> made to ensure that these files are small.  For example, they don't
>> need to contain a very large number of points.
>>
>>> In a way, the best approach would be to create a checkpoint file
>>> from a test parameter file and store this.  This would be the most
>>> efficient way to store the complete information.  Unfortunately,
>>> Cactus test cases support only ASCII output (so far?).
>>
>> I think this would store much too much information.  Yes, it would be
>> good if the test suite mechanism supported hdf5 files.  Anyone
>> interested in adding this might look into the h5diff tool...
>>
>> -- 
>> Ian Hinder
>> ian.hinder at aei.mpg.de
>>
>> _______________________________________________
>> Users mailing list
>> Users at einsteintoolkit.org
>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users


-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/