[ET Trac] [Einstein Toolkit] #114: per-variable tolerances for Cactus testsuites
Einstein Toolkit
trac-noreply at einsteintoolkit.org
Fri Sep 30 07:21:06 CDT 2011
#114: per-variable tolerances for Cactus testsuites
--------------------------+-------------------------------------------------
Reporter: knarf | Owner: tbode
Type: enhancement | Status: review
Priority: major | Milestone: ET_2011_10
Component: Cactus | Version:
Resolution: | Keywords:
--------------------------+-------------------------------------------------
Comment (by tbode):
It would not be difficult at all to take only the first and last instance,
however I disagree that uniqueness requires unreadable expressions. You
might have to break it up into several lines that are more specific to
address subsets of the larger group. In the extreme case you would simply
enter the entire file name. It's certainly possible to make it both unique
and readable with a minimal familiarity of regular expressions.
The problem with taking the first or last instance is if the thorn
developer adds an expression which matches more than was intended,
increasing the tolerance of a secondary data file to the point where it's
no longer a good test. There's no verboseness option available in test.ccl
to developers. Any warnings put into the RunTest scripts to alert the
developer would warn everybody, every time they run the test.
If you insist going with first or last instance, then I would also insist
we include an option to print a tolerance table for a test, where the
tolerances for every available datafile in the test are tabulated. That
way a developer can assess the success of their expression set without
having to modify the RunTest*.pl perl scripts. For some tests this would
apparently require 200+ lines .... might not actually be a bad addition
either way actually.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/114#comment:9>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
More information about the Trac
mailing list