#1872: test system does not detect extra lines in output files -----------------------+---------------------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: critical | Milestone: ET_2016_11 Component: Cactus | Version: development version Keywords: testsuite | -----------------------+---------------------------------------------------- I accidentally removed some lines of output from the stored test suite data in git hash [https://bitbucket.org/eloisa/ctthorns/commits/79c5e0d32fb1e3d6b219d7b55211fa... 79c5e0d32fb1e3d6b219d7b55211fa3c388f7268] of ctthorns in the files {{{CT_MultiLevel/test/constraints_spherical/*_norm_eqn?.asc}}}. Yet the test all passed. I have since restored the changed files.
This I would consider quite serious since extra output lines should always cause the test to fail rather than being silently ignored.
I am marking this as critical since it potentially renders the test suites useless when detecting changes. If we fell we can "document away" this then, it can be downgraded to "major".