On 16 Jun 2010, at 04:34, Erik Schnetter wrote:

On Jun 14, 2010, at 11:07 , Ian Hinder wrote:

Hi all,

I have written some test cases for the McLachlan evolution code.

Yay! Thanks!

For the exact solution I only output 3D of the admbase gxx and kxx variables. If this is considered too large, we can omit the 3D HDF5 output for now, as it is not used in the current Cactus testsuite mechanism.

I would omit them. I did include a lot of test result output in IsolatedHorizon, and I received many complaints from people about the time it takes to download the thorn, and the disk space it uses. I don't really agree with these sentiments, but I now think that extensive output should be stored somewhere else -- maybe in a special repository that one checks out only when test cases fail.

I agree - I also don't like to have to download a large amount of testsuite data, or have it taking up home directory quota space. If we have 200 thorns, and each of them are using 10 MB for testsuites, that is 2 GB per source tree.

I have modified the tests to output only 1D ASCII data in the x, y and z directions. This should be enough to catch regressions, since all the points on the grid will couple to each other in a single crossing time. The ML_BSSN_Test thorn is now 1.4 MB uncompressed (125 KB compressed)

I have put the test suites in the existing ML_BSSN_Helper thorn, as this seemed the easiest solution, since ML_BSSN is automatically generated.

Actually, the ML_BSSN_Helper thorns are also generated automatically since they are all very similar. Their source is in the "prototype" subdirectory. This process would overwrite your test cases. You could introduce a new thorn, e.g. ML_BSSN_Test.

I have done this now.

The test does not test the shift terms. For that I would use the shifted gauge wave solution, but as far as I know the harmonic shift gauge condition is not implemented in McLachlan.

The tests are run on 2 processes as CarpetIOASCII output is used, so the output depends on the number of processes. If output on 1 process is better, let me know.

I prefer output on 2 processors.

Good.

In principle, the number of OpenMP threads should not matter in this case. However, the tests seem to fail if I use OMP_NUM_THREADS=2 (or leave it unset on my dual-core laptop, which makes it use 2). The failure is:

Issuing mpirun -np 2 /Users/ian/Cactus/llama/exe/cactus_mcl /Users/ian/Cactus/llama/arrangements/McLachlan/ML_BSSN_Helper/test/gw3d_Exact_ord4_15.par

alp.norm1.asc: differences below tolerance on 1 lines
alp.norm2.asc: differences below tolerance on 1 lines
alp.sum.asc: substantial differences
   significant differences on 1 (out of 2) lines
   maximum absolute difference in column 3 is 1.00044417195022e-11
   maximum relative difference in column 3 is 2.96613722781153e-15

Failure: 12 files compared, 3 differ, 1 differ significantly

Files which differ strongly:
[1] alp.sum.asc

I don't know why this fails.

With N grid points, the absolute error in the sum reduction is N times larger than the absolute error in the average reduction. I usually disable this reduction operation (and some others) in test cases.

I have removed output for the reduction, but I now get differences above tolerance in the Hamiltonian constraint:

H.x.asc: substantial differences

significant differences on 6 (out of 46) lines

maximum absolute difference in column 13 is 3.46657468355827e-12

maximum relative difference in column 13 is 9.69181535413736e-11

(insignificant differences on 9 lines)

H.y.asc: substantial differences

significant differences on 7 (out of 46) lines

maximum absolute difference in column 13 is 2.2119042708546e-12

maximum relative difference in column 13 is 4.60112943759897e-11

(insignificant differences on 8 lines)

H.z.asc: substantial differences

significant differences on 14 (out of 60) lines

maximum absolute difference in column 13 is 3.00423574906006e-12

maximum relative difference in column 13 is 6.2493109449911e-11

(insignificant differences on 7 lines)

The evolved variables show differences below tolerance.

I attach the patch which contains the testsuites. I am inclined to commit it, and warn people to use OMP_NUM_THREADS=1 when running it. At least this gives us a direct working regression test for McLachlan. But we should understand why the Hamiltonian constraint is giving these differences.

If we want to include this test in the release, we will need to modify the einsteintoolkit.th thornlist to include the new thorn. The test runs only on 2 processes, as specified in the test.ccl file, due to the CarpetIOASCII output being dependent on the process decomposition. I have run the test successfully on my laptop and on damiana, but not on anything more exotic.