#1872: test system does not detect extra lines in output files -----------------------+---------------------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: review Priority: critical | Milestone: ET_2016_11 Component: Cactus | Version: development version Resolution: | Keywords: testsuite -----------------------+----------------------------------------------------
Comment (by rhaas):
Looking at the actual file {{{lib/sbin/RunTestUtils.pl}}} this also does not seem to handle the case where the new file is too short properly. As far as I can tell a too-short new file results in its lines being assumed empty which perl will interpret as "all fields have value 0.". This may pass in case the expected data is small.
I think we need at least four new error types: 1. lines missing 1. extra lines 1. columns missing 1. extra columns all of which are strong failures.