<div dir="ltr">On Mon, Mar 20, 2017 at 11:19 AM, Roland Haas <span dir="ltr"><<a href="mailto:rhaas@illinois.edu" target="_blank">rhaas@illinois.edu</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Present: Frank, Eloisa, Erik, Steve, Roland, Peter, Vassili, Ian,<br>
Roberto<br>
<br>
Failing tests:<br>
* boostedpuncture failure likely due to code change<br>
* GRHydro tests shows different behaviour depending on options, the<br>
test seems to fail only with new enough compiler, and -ffast-math<br>
* affected hydro tests are very sensitive to data, it may be sufficient<br>
to use a mild increase in resolution<br>
* as a general approach we need to decide if we want -ffast-math or if<br>
removing it does not affect results too much. It may be sufficient<br>
for gcc to specify options that allow the same subset of fast-math<br>
that icc uses by default<br>
* if fast-math is needed for good performance, then we have to relax<br>
test constraints for these tests<br>
* action items:<br>
** run production benchmark with and without -fast-math on a modern<br>
machine (Erik), use GW150914<br></blockquote><div><br></div><div>Here are the results from Wheeler, after 20480 iterations (a few hours of run time):</div><div><br></div><div>With -Ofast (current state):</div><div><br></div><div><div> 100.0% 12981.7 0.0% Evolve 1.298e+04 1.102e+04 3.245e+13 5.409e+11 5.926e+12 1.079e+05 1.686e+12</div></div><div> 2.3% 298.5 13.6% | | | | |_ML_BSSN_EvolutionInteriorSplitBy1 264.3 262.5 6.609e+11 1.614e+07 2.598e+09 8.086e+06 1.585e+08<br></div><div><div> 3.3% 434.5 15.9% | | | | |_ML_BSSN_EvolutionInteriorSplitBy2 370.3 368.9 9.259e+11 2.26e+07 4.156e+09 1.133e+07 2.818e+08</div><div> 4.2% 543.7 24.0% | | | | |_ML_BSSN_EvolutionInteriorSplitBy3 428.3 426.6 1.071e+12 2.615e+07 4.976e+09 1.303e+07 2.968e+08</div><div><br></div></div><div>With -O3 -fno-math-errno -fno-trapping-math -fno-rounding-math -fno-signaling-nans -fcx-limited-range (only "harmless" optimizations):</div><div><br></div><div><div> 100.0% 12743.1 0.0% Evolve 1.274e+04 1.077e+04 3.186e+13 5.31e+11 5.817e+12 1.111e+05 1.631e+12</div></div><div> 2.5% 316.3 13.4% | | | | |_ML_BSSN_EvolutionInteriorSplitBy1 293.9 291.5 7.349e+11 1.794e+07 3.155e+09 8.642e+06 4.517e+08<br></div><div><div> 3.7% 477.0 15.3% | | | | |_ML_BSSN_EvolutionInteriorSplitBy2 413.6 412.3 1.034e+12 2.525e+07 4.774e+09 1.198e+07 3.414e+08</div><div> 4.7% 594.7 23.0% | | | | |_ML_BSSN_EvolutionInteriorSplitBy3 490.4 489.2 1.226e+12 2.993e+07 5.635e+09 1.478e+07 4.406e+08</div></div><div><br></div><div>This means:</div><div>- the RHS routines are about 8% slower</div><div>- the overall run time is 2% slower</div><div><br></div><div>The overall run times could also be affected by other random factors (e.g. I/O, network bandwidth, etc.), but the pure RHS numbers should be correct. However, since the overall run time increase is consistent with the RHS run time increase (with the usual other costs, e.g. ADM variables, horizon finding, mesh refinement, ...), I think they are reliable.</div><div><br></div><div>-erik</div><div><br></div></div>-- <br><div class="gmail_signature">Erik Schnetter <<a href="mailto:schnetter@cct.lsu.edu" target="_blank">schnetter@cct.lsu.edu</a>><br><a href="http://www.perimeterinstitute.ca/personal/eschnetter/" target="_blank">http://www.perimeterinstitute.ca/personal/eschnetter/</a></div>
</div></div>