Not sure if it is useful for detecting errors or for finding the source of a problem once a problem is detected, but we could keep larger files or complete data sets remotely on a server.
On Apr 20, 2010, at 4:35 AM, Ian Hinder wrote:
On 19 Apr 2010, at 16:33, Erik Schnetter wrote:
On Apr 19, 2010, at 2:43 , Ian Hinder wrote:
On 19 Apr 2010, at 05:21, Baiotti Luca wrote:
Hi Bruno,
the idea of the testsuites is to have reliable data (corresponding to a given parfile) to compare with, when one wants to check that the code works , e.g., on a new machine or that a change in the source code has not introduced unwanted behaviour. So the data are a fundamental part of the testsuites.
While I completely agree with what Luca said, I would like to echo Bruno's concern that some of the test suite data is very large. Since it is only designed for regression testing, there is no reason I can see to have such large data files. There is either a regression or there isn't, and once you have found that it is present, you can then do more detailed runs with more output to locate it. You shouldn't need a large amount of output.
If a thorn calculates N quantities, then ideally there should be N 3D output files, so that all quantities are tested. It is impossible to test correctness if there are only norms, and if there is only 1D output then problems off-axis are not detected.
I agree about norms and 1D being insufficient, but effort should be made to ensure that these files are small. For example, they don't need to contain a very large number of points.
In a way, the best approach would be to create a checkpoint file from a test parameter file and store this. This would be the most efficient way to store the complete information. Unfortunately, Cactus test cases support only ASCII output (so far?).
I think this would store much too much information. Yes, it would be good if the test suite mechanism supported hdf5 files. Anyone interested in adding this might look into the h5diff tool...
-- Ian Hinder ian.hinder@aei.mpg.de
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users