[ET Trac] [Einstein Toolkit] #899: Push testsuite results to a git repository

[Einstein Toolkit]

#899: Push testsuite results to a git repository
--------------------------+-------------------------------------------------
  Reporter:  eschnett     |       Owner:     
      Type:  enhancement  |      Status:  new
  Priority:  major        |   Milestone:     
 Component:  Cactus       |     Version:     
Resolution:               |    Keywords:     
--------------------------+-------------------------------------------------

Comment (by barry.wardell):

 Replying to [comment:3 hinder]:
 > >Would it make sense to have each test run as a single simulation with
 multiple restarts? This would fit better into the Simfactory scheme, but
 would prevent running them in parallel.
 >
 > At the moment, compilation on Datura takes about 10 minutes (on a good
 day), and each test run also takes about 10 minutes.  So yes, it would be
 useful to run them in parallel (30 mins down to 20 mins), but not
 essential.  Performance on other machines will vary, but I expect the
 compilation time to dominate (we have a relatively fast home filesystem).
 This is, of course, an abuse of the simfactory restart mechanism.  We
 could alternatively teach simfactory to run multiple test sets in a single
 simulation.  It already has special code for dealing with test suites.
 This would need some tweaking of mpirun/nprocs settings.

 Have you tried using ccache to speed up the compile process? It can make a
 surprisingly big difference to compile times in some cases. It might be
 that we are so filesystem limited that it's not worth it, but it's worth a
 try and much easier than trying to implement parallel tests.

 Replying to [comment:3 hinder]:
 > Having separate heads would indeed enable run expiry.  Why would tools
 get confused?  What we could also do is start a new head every time the
 tests pass successfully.  We are not usually interested in history before
 that point any more.  We could then expire old heads periodically if we
 wanted to.

 > Or just starting a new repository in a state where all the tests pass,
 which could correspond to a 6-monthly release.

 There isn't really any difference between having separate heads in the
 same repository and having separate repositories other than the
 convenience of being able to diff between revisions in a single
 repository. It's also straightforward to go from the separate heads case
 to the separate repositories case and back again.

 Replying to [comment:3 hinder]:
 > Having test results as separate commits in the same branch allowed me to
 detect this sort of behaviour.

 You could still detect this without the commits having to be on the same
 branch, couldn't you?

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/899#comment:4>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit