On Thu, May 12, 2016 at 08:00:40PM +0200, Eloisa Bentivegna wrote:
this is really due to the fact that this specific variable
(ct_multilevel-err) should not be tested for relative changes, as it's
difficult to define a natural tolerance, and sometimes even the smallest
differences in execution lead to very large relative changes. I have
just pushed a commit that makes sure that a failure is only triggered if
the absolute difference is above threshold.
That is great, thanks.
On the other hand, the actual change that most likely triggered this
was a change in option lists on Jenkins, and there most likely the
-Ofast option. Before build #758 [1], the Jenkins VM used a special
jenkins option list, and starting with that build, it now uses the
generic Ubuntu option list.
Now, the tests should obviously also pass using that option list, but I
found it interesting to know why it started to fail when it did. Was the
change in option list intentional?
Hi Frank,
No, this was not intentional. The change in question (
https://bitbucket.org/ianhinder/cactusjenkins/commits/d7021a52bd83448db589b2346c43441682eecabb) was not supposed to change the optionlist used for the main ET job, but now that I look at it, I see that it had exactly that effect. I think when I first made the change, I was working on a separate copy of the repository that wouldn't affect the main job, but when I committed it a few days later, I forgot that I needed to add an exclusion if it was in the main job.
I could add such an exclusion, but since it seems to work now and it is better to test the main ubuntu optionlist, I think I will just leave it as-is. Do you agree?
This will also have the effect of making the builds slower, as the standard ubuntu optionlist does not use ccache.