[ET Trac] [Einstein Toolkit] #1872: test system does not detect extra lines in output files

[Einstein Toolkit]

#1872: test system does not detect extra lines in output files
-----------------------+----------------------------------------------------
  Reporter:  rhaas     |       Owner:                     
      Type:  defect    |      Status:  review             
  Priority:  critical  |   Milestone:  ET_2016_11         
 Component:  Cactus    |     Version:  development version
Resolution:            |    Keywords:  testsuite          
-----------------------+----------------------------------------------------

Comment (by rhaas):

 Looking at the actual file {{{lib/sbin/RunTestUtils.pl}}} this also does
 not seem to handle the case where the new file is too short properly. As
 far as I can tell a too-short new file results in its lines being assumed
 empty which perl will interpret as "all fields have value 0.". This may
 pass in case the expected data is small.

 I think we need at least four new error types:
   1. lines missing
   1. extra lines
   1. columns missing
   1. extra columns
 all of which are strong failures.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1872#comment:6>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit