On Mon, Mar 20, 2017 at 11:19 AM, Roland Haas rhaas@illinois.edu wrote:
Present: Frank, Eloisa, Erik, Steve, Roland, Peter, Vassili, Ian, Roberto
Failing tests:
- boostedpuncture failure likely due to code change
- GRHydro tests shows different behaviour depending on options, the test seems to fail only with new enough compiler, and -ffast-math
- affected hydro tests are very sensitive to data, it may be sufficient to use a mild increase in resolution
- as a general approach we need to decide if we want -ffast-math or if removing it does not affect results too much. It may be sufficient for gcc to specify options that allow the same subset of fast-math that icc uses by default
- if fast-math is needed for good performance, then we have to relax test constraints for these tests
- action items:
** run production benchmark with and without -fast-math on a modern machine (Erik), use GW150914
Here are the results from Wheeler, after 20480 iterations (a few hours of run time):
With -Ofast (current state):
100.0% 12981.7 0.0% Evolve 1.298e+04 1.102e+04 3.245e+13 5.409e+11 5.926e+12 1.079e+05 1.686e+12 2.3% 298.5 13.6% | | | | |_ML_BSSN_EvolutionInteriorSplitBy1 264.3 262.5 6.609e+11 1.614e+07 2.598e+09 8.086e+06 1.585e+08 3.3% 434.5 15.9% | | | | |_ML_BSSN_EvolutionInteriorSplitBy2 370.3 368.9 9.259e+11 2.26e+07 4.156e+09 1.133e+07 2.818e+08 4.2% 543.7 24.0% | | | | |_ML_BSSN_EvolutionInteriorSplitBy3 428.3 426.6 1.071e+12 2.615e+07 4.976e+09 1.303e+07 2.968e+08
With -O3 -fno-math-errno -fno-trapping-math -fno-rounding-math -fno-signaling-nans -fcx-limited-range (only "harmless" optimizations):
100.0% 12743.1 0.0% Evolve 1.274e+04 1.077e+04 3.186e+13 5.31e+11 5.817e+12 1.111e+05 1.631e+12 2.5% 316.3 13.4% | | | | |_ML_BSSN_EvolutionInteriorSplitBy1 293.9 291.5 7.349e+11 1.794e+07 3.155e+09 8.642e+06 4.517e+08 3.7% 477.0 15.3% | | | | |_ML_BSSN_EvolutionInteriorSplitBy2 413.6 412.3 1.034e+12 2.525e+07 4.774e+09 1.198e+07 3.414e+08 4.7% 594.7 23.0% | | | | |_ML_BSSN_EvolutionInteriorSplitBy3 490.4 489.2 1.226e+12 2.993e+07 5.635e+09 1.478e+07 4.406e+08
This means: - the RHS routines are about 8% slower - the overall run time is 2% slower
The overall run times could also be affected by other random factors (e.g. I/O, network bandwidth, etc.), but the pure RHS numbers should be correct. However, since the overall run time increase is consistent with the RHS run time increase (with the usual other costs, e.g. ADM variables, horizon finding, mesh refinement, ...), I think they are reliable.
-erik