#2200: some test may be non-deterministic ---------------------------------+-------------------- Reporter: Roland Haas | Type: defect Status: new | Priority: unset Milestone: | Component: Other Version: development version | Keywords: ---------------------------------+-------------------- For the ET_2018_09 release it seems that tests run on Sep 6th and Sept 22nd differ and these tests show a different number of failed tests:
* comet__1_24.log * cori__1_16.log * osx-homebrew__2_2.log * osx-macports__2_2.log
and given that there should not have been a change in code between those days this seems suspicious.
The particular tests that changed are:
{{{ +++ b/results/comet__1_24.log @@ -1434,7 +1434,7 @@ AHFinderDirect: misner1.2-025 Success: 55 files compared, 9 differ in the last digits AHFinderDirect: recoverML-EE - Success: 5 files compared, 5 differ in the last digits + Failure: 2 files missing, 3 files compared, 3 differ, 3 differ significantly Carpet: 64k2 Success: 0 files identical Carpet: test_restrict_sync }}}
{{{ +++ b/results/cori__1_16.log @@ -1446,7 +1446,7 @@ CarpetIOHDF5: CarpetWaveToyNewRecover_test_1proc Success: 12 files compared, 7 differ in the last digits CarpetIOHDF5: CarpetWaveToyRecover_test_1proc - Failure: 12 files missing, 0 files compared, 0 differ + Success: 12 files compared, 7 differ in the last digits CarpetIOHDF5: CarpetWaveToyRecover_test_newcp_1proc Success: 12 files compared, 7 differ in the last digits CarpetIOHDF5: newsep }}}
{{{ +++ b/results/osx-homebrew__2_2.log @@ -1631,7 +1631,7 @@ PeriodicCarpet: testperiodicinterp Success: 1 files identical QuasiLocalMeasures: qlm-bl - Failure: 54 files missing, 115 files compared, 115 differ + Success: 169 files compared, 143 differ in the last digits QuasiLocalMeasures: qlm-ks Success: 169 files compared, 138 differ in the last digits QuasiLocalMeasures: qlm-ks-EE }}}
{{{ +++ b/results/osx-macports__2_2.log @@ -1631,7 +1631,7 @@ PeriodicCarpet: testperiodicinterp Success: 1 files identical QuasiLocalMeasures: qlm-bl - Failure: 54 files missing, 115 files compared, 115 differ + Success: 169 files compared, 144 differ in the last digits QuasiLocalMeasures: qlm-ks Success: 169 files compared, 136 differ in the last digits QuasiLocalMeasures: qlm-ks-EE }}}