[Users] Failing test cases

[Ian Hinder]

On 2 Dec 2011, at 00:25, Ian Hinder wrote:

> 
> On 1 Dec 2011, at 23:51, Erik Schnetter wrote:
> 
>> I just see that, on my local system, 7 out of 127 test cases are
>> failing. Do we know why? I sincerely hope I'm not to blame for too
>> many of those 7...
> 
> Unfortunately due to computer issues here at the AEI, the automated tests (http://damiana2.aei.mpg.de/~ianhin/testsuites/einsteintoolkit/) didn't run between 9th and 25th of November.  All the currently-failing tests started failing some time between those dates.
> 
> There are timer files committed to the test directory of the CarpetIOASCII tests which will cause the tests to fail, but the tests are also failing due to other problems.

I have performed a bisection search of the history of the ET between those two dates, and have identified the following commits to McLachlan as the culprit for the first failure of RotatingSymmetry180/KerrSchild-rotating-180.par:

> commit b3a25f307c9bb55370380b97e6074df1a30f645d
> Author: Ian Hinder <ian.hinder at aei.mpg.de>
> Date:   Sun Nov 20 13:23:59 2011 +0100
> 
>     Regenerate ML_ADMConstraints
> 
> commit 5fd1f73f9ca35f0852c0cb61f77abc8b053f5aff
> Author: Ian Hinder <ian.hinder at aei.mpg.de>
> Date:   Thu Nov 17 17:47:43 2011 +0100
> 
>     McLachlan_ADMConstraints.m: Fix index errors
> 

This thorn computes the Einstein constraints using the ADM variables, and I was working on it recently because I was doing tests with initial data only and hence didn't want to use the BSSN constraints.  I noticed that the Ricci tensor used to compute the constraints was computed incorrectly, and fixed this.  I have tested the fix and I am confident that it is correct.  I should have realised that this would have an effect on the test suites!  Some test suites output these constraints, so now that the constraints are different, the tests fail.  On this commit, the only failures in this test are due to the constraint variables.  HOWEVER: the test run with the current ET fails also because of differences in the ADM variables, so there must be another problem as well.  I will continue investigating.

In case anyone is interested, I used an Einstein Toolkit Git super-repository to perform the bisection search.  Barry and I have been using this super-repository for about half a year.  The idea is to have a Git "super-repository" which contains pointers to the versions of all the ET components.  This means that a single commit ID in the super-repository can be used to identify a snapshot of the ET.  This repository is updated on every change to the ET, so you can easily check out any version with a single command.  For this to work, all the repositories have to be Git repositories, so Barry has set up git-svn mirrors of all the ET SVN repositories, and a mirror of the Carpet Mercurial repository.  Another use of this technique is for code-provenance.  You can in principle identify the code used to run a simulation via a single commit ID (plus maybe some local patches).  This is complementary to Formaline which stores the source in the simulation.  It allows you to identify differences at the level of commits and authors, rather than at the level of diffs between source trees.

-- 
Ian Hinder
http://numrel.aei.mpg.de/people/hinder