#702: Tests should use IO::out_fileinfo = "none" --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: review Priority: minor | Milestone: Component: Other | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by knarf):

* status: new => review

#702: Tests should use IO::out_fileinfo = "none" --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: reviewed_ok Priority: minor | Milestone: Component: Other | Version: Resolution: | Keywords: --------------------------+-------------------------------------------------

Comment (by hinder):

Replying to [comment:1 eschnett]:

Please go ahead. Ideally, you would generate the new test case output by

removing lines from the old output, but that may be too much to ask for.

Since the changes should be invisible to the Cactus test mechanism, it should be enough to rerun the tests with the new output, verify that they still pass, then commit the new data. We can then use version-control tools to examine the diff, which should consist only of these header lines.

#702: Tests should use IO::out_fileinfo = "none" --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: reviewed_ok Priority: minor | Milestone: Component: Other | Version: Resolution: | Keywords: --------------------------+-------------------------------------------------

Comment (by rhaas):

Replying to [comment:4 hinder]:

Replying to [comment:1 eschnett]:

...

Please go ahead. Ideally, you would generate the new test case output

by removing lines from the old output, but that may be too much to ask for.

Since the changes should be invisible to the Cactus test mechanism, it

should be enough to rerun the tests with the new output, verify that they still pass, then commit the new data. We can then use version-control tools to examine the diff, which should consist only of these header lines.

I would actually be very surprised if the differences were only the header lines. Some of those test files were generated years ago with a compiler and options that are no longer in use. I would very much expect that regenerating the data would change the data in the files (this already happens when comparing intel and gcc compiler results).

#702: Tests should use IO::out_fileinfo = "none" --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: closed Priority: minor | Milestone: Component: Other | Version: Resolution: fixed | Keywords: --------------------------+-------------------------------------------------

Comment (by knarf):

Replying to [comment:6 rhaas]:

Replying to [comment:4 hinder]:

...

Replying to [comment:1 eschnett]:

...

Please go ahead. Ideally, you would generate the new test case

output by removing lines from the old output, but that may be too much to ask for.

...

Since the changes should be invisible to the Cactus test mechanism, it

should be enough to rerun the tests with the new output, verify that they still pass, then commit the new data. We can then use version-control tools to examine the diff, which should consist only of these header lines.

I would actually be very surprised if the differences were only the

header lines. Some of those test files were generated years ago with a compiler and options that are no longer in use. I would very much expect that regenerating the data would change the data in the files (this already happens when comparing intel and gcc compiler results).

Are we here talking about tests that haven't run in a while? All the ones that are regularly run should still pass after this change, shouldn't they? They should be tested, using current compilers, all the time, and except the ioinfo, none of the generated data should change _by this source change alone_.

#702: Tests should use IO::out_fileinfo = "none" --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: closed Priority: minor | Milestone: Component: Other | Version: Resolution: fixed | Keywords: --------------------------+-------------------------------------------------

Comment (by knarf):

Yes, that is true. It even likely would look different on different machines, or using different compilers or compiler options. And this could even lead to new test suite failures, on some machines.