On Thu, May 12, 2016 at 08:00:40PM +0200, Eloisa Bentivegna wrote:
this is really due to the fact that this specific variable (ct_multilevel-err) should not be tested for relative changes, as it's difficult to define a natural tolerance, and sometimes even the smallest differences in execution lead to very large relative changes. I have just pushed a commit that makes sure that a failure is only triggered if the absolute difference is above threshold.
That is great, thanks.
On the other hand, the actual change that most likely triggered this was a change in option lists on Jenkins, and there most likely the -Ofast option. Before build #758 [1], the Jenkins VM used a special jenkins option list, and starting with that build, it now uses the generic Ubuntu option list.
Now, the tests should obviously also pass using that option list, but I found it interesting to know why it started to fail when it did. Was the change in option list intentional?
Frank
[1] https://build.barrywardell.net/job/EinsteinToolkit/lastCompletedBuild/testRe...