[Users] Test failures

[Ian Hinder]

On 7 Nov 2013, at 20:36, Frank Löffler <knarf at cct.lsu.edu> wrote:

> Hi
> 
> So far, Ian ran tests on datura and I ran the testsuites, on my
> workstation. The gcc build shows no failures at all, but the intel suite
> shows 9, but in part different than the one on datura.
> 
> http://einsteintoolkit.org/release-info/parse_testsuite_results.php

…

> Some of the Multipole tests fail due to tolerance in the convergence
> order output. I would like to see someone who is expert in that
> thorn/testsuite to take a look.

This is a new test, and it's just a tolerance issue. I will increase the tolerance.

> So, in short: apart from a hydro problem on datura we see the 'usual'
> Exact-related problems, a failure in balsara4_1d and in the testsuites
> of Multipole.
> 
> However, we need more results. Please test other machines as well! 

The tests are now run automatically on Datura.  The results are sent to this Jenkins job:

	https://build.barrywardell.net/view/All/job/EinsteinToolkitDatura/

where you can browse the reasons for the test failures under "Latest test result".  The output and diff files are there.  
The test summary log file is automatically committed to the ET release-info repository; I can disable this after the release, otherwise there will be too many commit messages to the commits list.

I run the tests from a cron job on the head node and push the results to a repository (http://git.barrywardell.net/?p=EinsteinToolkitDaturaTestResults.git;a=tree).  This means it can run unattended, without having to leave a connection open to the cluster, or giving a web application (Jenkins) access to the cluster. On the other hand, it means a bit of setup on each cluster, and for cron to be available.

The (131 line) script which runs the tests is here:

	https://bitbucket.org/ianhinder/cactusjenkins/src/master/build-and-test?at=master

It should be run from an existing Cactus directory.  It currently assumes that the Cactus directory was checked out from the Git super-repository using

	git clone --recursive git://git.barrywardell.net/EinsteinToolkit.git

but only the section labelled "Update" assumes this; you could replace that section with a call to update using GetComponents instead (patches welcome!).  Other customisations should be fairly obvious.  It also assumes that the cactusjenkins repo is in repos/cactusjenkins, and the release-info repo is in ~/release-info as a git-svn checkout.  If you want to use it on your cluster, and would like me to create the corresponding Jenkins job on build.barrywardell.net to display the results, please let me know.

-- 
Ian Hinder
http://numrel.aei.mpg.de/people/hinder

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20131112/9b79e24c/attachment.bin