[Users] McLachlan test suite

[Erik Schnetter]

On Jun 14, 2010, at 11:07 , Ian Hinder wrote:

> Hi all,
>
> I have written some test cases for the McLachlan evolution code.

Yay!  Thanks!

> This thorn is currently used in other test suites, but it doesn't  
> have one of its own.  The minimal test I have written is a 3D  
> diagonal gauge wave in unigrid with periodic boundary conditions run  
> for one crossing time, so mostly it tests that the RHSs are  
> correct.  There are two resolutions (15^3 and 30^3) as well as the  
> exact solution as computed by Exact, so from the test suite output  
> you can compute convergence to test that everything is correct (the  
> automatic regression testsuite mechanism does not do this though).   
> I attach the parameter files for comments.  I have tested 4th order  
> convergence to the exact solution.
>
> I have enabled 3D HDF5 output in the test suite in anticipation that  
> we might want to have HDF5 comparisons added at some point, and so  
> that when investigating failures there is full information  
> available.  The output is from the initial data and the last  
> iteration only, with full compression.
>
> The test output sizes are:
>
> n = 15		2.9 M
> n = 30		7.9 M
> Exact		100 kB
>
> Total: 11 M
>
> For the exact solution I only output 3D of the admbase gxx and kxx  
> variables.  If this is considered too large, we can omit the 3D HDF5  
> output for now, as it is not used in the current Cactus testsuite  
> mechanism.

I would omit them.  I did include a lot of test result output in  
IsolatedHorizon, and I received many complaints from people about the  
time it takes to download the thorn, and the disk space it uses.  I  
don't really agree with these sentiments, but I now think that  
extensive output should be stored somewhere else -- maybe in a special  
repository that one checks out only when test cases fail.

> I have put the test suites in the existing ML_BSSN_Helper thorn, as  
> this seemed the easiest solution, since ML_BSSN is automatically  
> generated.

Actually, the ML_BSSN_Helper thorns are also generated automatically  
since they are all very similar.  Their source is in the "prototype"  
subdirectory.  This process would overwrite your test cases.  You  
could introduce a new thorn, e.g. ML_BSSN_Test.

> The test does not test the shift terms.  For that I would use the  
> shifted gauge wave solution, but as far as I know the harmonic shift  
> gauge condition is not implemented in McLachlan.
>
> The tests are run on 2 processes as CarpetIOASCII output is used, so  
> the output depends on the number of processes.  If output on 1  
> process is better, let me know.

I prefer output on 2 processors.

> In principle, the number of OpenMP threads should not matter in this  
> case.  However, the tests seem to fail if I use OMP_NUM_THREADS=2  
> (or leave it unset on my dual-core laptop, which makes it use 2).   
> The failure is:
>
> Issuing mpirun -np 2 /Users/ian/Cactus/llama/exe/cactus_mcl /Users/ 
> ian/Cactus/llama/arrangements/McLachlan/ML_BSSN_Helper/test/ 
> gw3d_Exact_ord4_15.par
>
>  alp.norm1.asc: differences below tolerance on 1 lines
>  alp.norm2.asc: differences below tolerance on 1 lines
>  alp.sum.asc: substantial differences
>     significant differences on 1 (out of 2) lines
>     maximum absolute difference in column 3 is 1.00044417195022e-11
>     maximum relative difference in column 3 is 2.96613722781153e-15
>
> Failure: 12 files compared, 3 differ, 1 differ significantly
>
> Files which differ strongly:
>   [1] alp.sum.asc
>
> I don't know why this fails.


With N grid points, the absolute error in the sum reduction is N times  
larger than the absolute error in the average reduction.  I usually  
disable this reduction operation (and some others) in test cases.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/