[Users] RotatingDBHIVP/E2xeon_test_rdbh on stampede

[Erik Schnetter]

Ian

In several places, our OpenMP parallelization is not deterministic. For example, LoopControl (and some other thorns) allocate thread work items dynamically. However, this should not influence the results.

Carpet's reduction algorithm is not deterministic. The order in which the partial results are summed up may lead to different round-off errors in norms. However, here the grid points values differ, which can't be explained by this.

Maybe there was a change to this thorn after I ran this test?

-erik

On Nov 22, 2013, at 14:49 , Ian Hinder <ian.hinder at aei.mpg.de> wrote:

> Hi Erik,
> 
> For me, the test
> 
> RotatingDBHIVP/E2xeon_test_rdbh fails on stampede, but for you it passes on stampede.  The optionlists seem to be the same. Can you think of a possible explanation?  The results should be the same.
> 
> Your results: http://git.barrywardell.net/?p=EinsteinToolkitTestResults-sandbox.git;a=blob;f=build-and-test_2/output-0000/TEST/sim/RotatingDBHIVP/E2xeon_test_rdbh.diffs;h=121755eeae5e02e73dfd63daaf3b34ae94f33cd7;hb=refs/heads/stampede
> 
> My results: http://git.barrywardell.net/?p=EinsteinToolkitDaturaTestResults.git;a=blob;f=build-and-test_2/output-0000/TEST/sim/RotatingDBHIVP/E2xeon_test_rdbh.diffs;h=5ab486fce91ab197007d2b80ba008d2b7166211d;hb=c1537ed07b873c3933d1da4ad53dac5ce846b4b6
> 
> (the second repo is badly-named; the result is from stampede not datura)
> 
> I have diffed the output log files, and there is nothing suspicious. Same number of OpenMP threads, at least for the 2 proc version.  I'm wondering if the differences could come down to memory alignment differences between your executable and mine, due to different strings (dates, home directories etc) being present in the executable.  See http://software.intel.com/en-us/articles/run-to-run-reproducibility-of-floating-point-calculations-for-applications-on-intel-xeon.  
> 
> The largest absolute difference measured is 2.92969914994501e-09.
> 
> -- 
> Ian Hinder
> http://numrel.aei.mpg.de/people/hinder
> 

-- 
Erik Schnetter <eschnetter at perimeterinstitute.ca>
http://www.perimeterinstitute.ca/personal/eschnetter/

My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu/.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20131122/bd8a7084/attachment.bin