[Users] automatic check of testsuites?

[Ian Hinder]

On 1 Apr 2011, at 09:25, Luca Baiotti wrote:

> On 3/29/11 6:54 PM, Ian Hinder wrote:
>> 
>> On 29 Mar 2011, at 10:09, Alexander Beck-Ratzka wrote:
>> 
>>> On Tuesday, March 29, 2011 09:57:23 Ian Hinder wrote:
>>>> On 29 Mar 2011, at 03:01, Luca Baiotti wrote:
>>>>> Hello,
>>>>> 
>>>>> by reading the minutes I started wondering whether there is
>>>>> again a mechanism to check the testsuites regularly and
>>>>> automatically on different machines, like the mechanism that
>>>>> Thomas Radke had created. I read briefly the Einstein Toolkit
>>>>> website, but did not find this mentioned.
>>>> 
>>>> I have set up something which runs nightly on my workstation and
>>>> tests the ET thornlist with both versions of Carpet as well as
>>>> some of our private thorns.  It is by no means perfect and would
>>>> need quite a bit of work to make it useful.  We should have
>>>> something "official" which runs on each machine.  I suspect one
>>>> barrier to working on this is the hope that simfactory will one
>>>> day be able to run testsuites, and that that would be the best
>>>> infrastructure to use for such a project.  Unfortunately, nobody
>>>> has the time to put the required work in.  Are you volunteering?
>>>> :)
>>> 
>>> 
>>> I would be interested in doing this. However, I would need some
>>> instructions...
>> 
>> What we really need is for somebody to take charge of the project and
>> push it forward, so if you can do that it would be great!  I'm sure
>> there are a few people in the same situation as me who don't have
>> time to devote to making this happen, but can give advice and help.
>> We could start by getting some ideas together about what such a
>> system would entail, and whether there are existing systems which we
>> can either build on or use as inspiration (e.g. Thomas' system from
>> before, or something like "buildbot").
>> 
>> I have just made a new page on the ET Wiki for this project and
>> written some initial notes and requirements:
>> 
>> https://docs.einsteintoolkit.org/et-docs/Automated_testing
>> 
>> It would be good if interested people could look at this page and
>> add/edit what is there.  We can discuss on the mailing list.
> 
> A first question: how does one run the testsuites (the Cactus testsuites 
> in the present form) on a queueing system?

At the end of your job submission script, you need to replace the usual mpirun command with a command to run the testsuites.You have to write a custom submission script.  For example, on Kraken, I replace the usual aprun (cf. mpirun) command at the end of the submission script with:

	cd Cactus
	export CCTK_TESTSUITE_RUN_COMMAND="aprun -n \$nprocs -d 1 \$exe \$parfile"
	export CCTK_TESTSUITE_RUN_PROCESSORS=1
	make sim-testsuite PROMPT=no

This avoids the need for interactive prompting.  The output is then in the TEST directory.  One complication is that the test suites need access to the Cactus source directory (for the test parameter files).  On some clusters, e.g. Kraken, a user's home directory is not accessible from the compute nodes. In that case, you have to copy your Cactus configuration to the scratch filesystem usually used for data output.

-- 
Ian Hinder
ian.hinder at aei.mpg.de