[Users] McLachlan test suite

Wed Jun 16 09:11:20 CDT 2010

On 16 Jun 2010, at 04:34, Erik Schnetter wrote:

> On Jun 14, 2010, at 11:07 , Ian Hinder wrote:
>
>> Hi all,
>>
>> I have written some test cases for the McLachlan evolution code.
>
> Yay!  Thanks!
>
>> For the exact solution I only output 3D of the admbase gxx and kxx  
>> variables.  If this is considered too large, we can omit the 3D  
>> HDF5 output for now, as it is not used in the current Cactus  
>> testsuite mechanism.
>
> I would omit them.  I did include a lot of test result output in  
> IsolatedHorizon, and I received many complaints from people about  
> the time it takes to download the thorn, and the disk space it  
> uses.  I don't really agree with these sentiments, but I now think  
> that extensive output should be stored somewhere else -- maybe in a  
> special repository that one checks out only when test cases fail.

I agree - I also don't like to have to download a large amount of  
testsuite data, or have it taking up home directory quota space.  If  
we have 200 thorns, and each of them are using 10 MB for testsuites,  
that is 2 GB per source tree.

I have modified the tests to output only 1D ASCII data in the x, y and  
z directions.  This should be enough to catch regressions, since all  
the points on the grid will couple to each other in a single crossing  
time.  The ML_BSSN_Test thorn is now 1.4 MB uncompressed (125 KB  
compressed)

>> I have put the test suites in the existing ML_BSSN_Helper thorn, as  
>> this seemed the easiest solution, since ML_BSSN is automatically  
>> generated.
>
> Actually, the ML_BSSN_Helper thorns are also generated automatically  
> since they are all very similar.  Their source is in the "prototype"  
> subdirectory.  This process would overwrite your test cases.  You  
> could introduce a new thorn, e.g. ML_BSSN_Test.

I have done this now.

>> The test does not test the shift terms.  For that I would use the  
>> shifted gauge wave solution, but as far as I know the harmonic  
>> shift gauge condition is not implemented in McLachlan.
>>
>> The tests are run on 2 processes as CarpetIOASCII output is used,  
>> so the output depends on the number of processes.  If output on 1  
>> process is better, let me know.
>
> I prefer output on 2 processors.

Good.

>> In principle, the number of OpenMP threads should not matter in  
>> this case.  However, the tests seem to fail if I use  
>> OMP_NUM_THREADS=2 (or leave it unset on my dual-core laptop, which  
>> makes it use 2).  The failure is:
>>
>> Issuing mpirun -np 2 /Users/ian/Cactus/llama/exe/cactus_mcl /Users/ 
>> ian/Cactus/llama/arrangements/McLachlan/ML_BSSN_Helper/test/ 
>> gw3d_Exact_ord4_15.par
>>
>> alp.norm1.asc: differences below tolerance on 1 lines
>> alp.norm2.asc: differences below tolerance on 1 lines
>> alp.sum.asc: substantial differences
>>    significant differences on 1 (out of 2) lines
>>    maximum absolute difference in column 3 is 1.00044417195022e-11
>>    maximum relative difference in column 3 is 2.96613722781153e-15
>>
>> Failure: 12 files compared, 3 differ, 1 differ significantly
>>
>> Files which differ strongly:
>>  [1] alp.sum.asc
>>
>> I don't know why this fails.
>
>
> With N grid points, the absolute error in the sum reduction is N  
> times larger than the absolute error in the average reduction.  I  
> usually disable this reduction operation (and some others) in test  
> cases.

I have removed output for the reduction, but I now get differences  
above tolerance in the Hamiltonian constraint:

    H.x.asc: substantial differences
       significant differences on 6 (out of 46) lines
       maximum absolute difference in column 13 is 3.46657468355827e-12
       maximum relative difference in column 13 is 9.69181535413736e-11
       (insignificant differences on 9 lines)
    H.y.asc: substantial differences
       significant differences on 7 (out of 46) lines
       maximum absolute difference in column 13 is 2.2119042708546e-12
       maximum relative difference in column 13 is 4.60112943759897e-11
       (insignificant differences on 8 lines)
    H.z.asc: substantial differences
       significant differences on 14 (out of 60) lines
       maximum absolute difference in column 13 is 3.00423574906006e-12
       maximum relative difference in column 13 is 6.2493109449911e-11
       (insignificant differences on 7 lines)

The evolved variables show differences below tolerance.

I attach the patch which contains the testsuites.  I am inclined to  
commit it, and warn people to use OMP_NUM_THREADS=1 when running it.   
At least this gives us a direct working regression test for  
McLachlan.  But we should understand why the Hamiltonian constraint is  
giving these differences.

If we want to include this test in the release, we will need to modify  
the einsteintoolkit.th thornlist to include the new thorn.  The test  
runs only on 2 processes, as specified in the test.ccl file, due to  
the CarpetIOASCII output being dependent on the process  
decomposition.  I have run the test successfully on my laptop and on  
damiana, but not on anything more exotic.

-- 
Ian Hinder
ian.hinder at aei.mpg.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20100616/9b22d66d/attachment-0002.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Add-test-cases-for-BSSN-3D-diagonal-gauge-wave.patch.gz
Type: application/x-gzip
Size: 127545 bytes
Desc: not available
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20100616/9b22d66d/attachment-0001.gz 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20100616/9b22d66d/attachment-0003.html 