On Apr 19, 2010, at 2:43 , Ian Hinder wrote:
On 19 Apr 2010, at 05:21, Baiotti Luca wrote:
Hi Bruno,
the idea of the testsuites is to have reliable data (corresponding to a given parfile) to compare with, when one wants to check that the code works , e.g., on a new machine or that a change in the source code has not introduced unwanted behaviour. So the data are a fundamental part of the testsuites.
While I completely agree with what Luca said, I would like to echo Bruno's concern that some of the test suite data is very large. Since it is only designed for regression testing, there is no reason I can see to have such large data files. There is either a regression or there isn't, and once you have found that it is present, you can then do more detailed runs with more output to locate it. You shouldn't need a large amount of output.
If a thorn calculates N quantities, then ideally there should be N 3D output files, so that all quantities are tested. It is impossible to test correctness if there are only norms, and if there is only 1D output then problems off-axis are not detected.
In a way, the best approach would be to create a checkpoint file from a test parameter file and store this. This would be the most efficient way to store the complete information. Unfortunately, Cactus test cases support only ASCII output (so far?).
-erik