[ET Trac] [Einstein Toolkit] #114: per-variable tolerances for Cactus testsuites

[Einstein Toolkit]

#114: per-variable tolerances for Cactus testsuites
--------------------------+-------------------------------------------------
  Reporter:  knarf        |       Owner:  tbode     
      Type:  enhancement  |      Status:  review    
  Priority:  major        |   Milestone:  ET_2011_11
 Component:  Cactus       |     Version:            
Resolution:               |    Keywords:            
--------------------------+-------------------------------------------------

Comment (by tbode):

 Thanks for looking at the patch.

 * Concerning debugging code, the only debugging left uncommented is in
 PrintDataBase, which is in there specifically ''for'' debugging and only
 for debugging.  This should be left in there so that the database printing
 routine descends one step lower in the hierarchy, giving useful
 information instead of just the reference to the hash. There is a
 PrintDataBase call in RunTest.pl that is commented out.  If we don't want
 any commented debugging, this should also be deleted.  If we want no
 debugging at all committed, that entire function should be removed.
 * I fully expected a complaint concerning the keywords. Easily changed.
 Long all-caps really annoy me so I prefer not to use it while coding.
 Search and replace are sufficient. I apologize for having left the case is
 is. Figured there would be enough discussion concerning functionality that
 there would be changes anyway. I didn't think of a better choice for the
 keyword, though, that wasn't absurdly long or misleading.
 * I implemented it as an extra keyword as it was easier to parse the lines
 this way, and specifying the expression first and tolerance second makes
 more sense to me as a user. Changing how ABSTOL and RELTOL are specified
 would outdate all current test.ccl files and would be a much more invasive
 change. I could put in extra loops of logic to treat both old and new
 cases.  I should think on this some more, whether there is a good way to
 handle this, but I prefer a second keyword.

 The interface in the patch ''does'' allow any number of expressions to be
 specified at either level. Where it falls short is in logic when a file
 matches more than one expression.  Currently any test tolerances override
 thorn-wide tolerances, and expression-specific tolerances blindly override
 all other tolerances.  If it matches more than one expression, though,
 it's only the last match's tolerance that is used. How should we treat
 cases where a filename matches more than one expression?  I'm inclining
 towards throwing an error if this happens indicating that test.ccl should
 be modified.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/114#comment:5>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit