[Einstein Toolkit] #2178: PITTNull can take a very long time to compile with the Intel compiler

List overview All Threads
Download

newer

older

[Einstein Toolkit] #2167: clean up...

[Einstein Toolkit] #1263:...

Einstein Toolkit

31 Jul 2018 31 Jul '18

3:40 p.m.

#2178: PITTNull can take a very long time to compile with the Intel compiler ---------------------------------+----------------------------------- Reporter: Roland Haas | Type: defect Status: new | Priority: minor Milestone: | Component: EinsteinToolkit thorn Version: development version | Keywords: NullConstr ---------------------------------+----------------------------------- Compiling on stampede2-skx (2018_02 simfactory files, so ifort 18.0.0) the file {{{NullConstr/src/NullConstr_R00.F90}}} takes a very long time (103 minutes an counting) to compile.

My guess is that one has to translate its array operations to loops (similar to what was done for other source files in PITTNull), otherwise the compiler just takes too long to try and optimize the code.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Show replies by date

Einstein Toolkit

2 Aug 2 Aug

8:31 a.m.

#2178: PITTNull can take a very long time to compile with the Intel compiler ------------------------------------+--------------------------------- Reporter: Roland Haas | Owner: (none) Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Resolution: | Keywords: NullConstr ------------------------------------+---------------------------------

Comment (by yosef@…):

The source looks like some automatic code generation output with very long expressions. Adding the directive "!DIR$ OPTIMIZE: 2" sped up compilation on my machine to just a few seconds. I didn't check the performance of the resulting code...

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:1 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

8:38 a.m.

Comment (by Roland Haas):

Thank you. Just to be sure: before adding the directive, compilation was slow?

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:2 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

10:59 a.m.

Comment (by yosef@…):

It was, but I didn't let it run to completion. I waited 25 minutes and then gave up.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:3 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

11:03 a.m.

Comment (by Roland Haas):

It also seems as if we already replaced array constructs with explicit loops in 388c945 - NullConstr: replace array operations by explicit loops (4 years, 9 months ago) and it seems is now the Intel compiler is finally smart enough to try and optimize the code so Yosef suggestion (maybe with an {{{#ifdef __INTEL__}}} around it) seems like the best solution.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:4 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

11:13 a.m.

Comment (by Roland Haas):

@Yosef: could I impose on you to check which other files are affected (by adding the optimization option to the ones that are slow one by one) and to prepare a patch, please? The {{{#ifdef}}} construct I mentioned would be this (note that this is doing the "right thing" even when a user specified {{{-O0}}} on the command line [see the docs linked below] "The procedure is compiled with an optimization level equal to the smaller of n and the optimization level specified by the O compiler option on the command line."): {{{ #ifdef __INTEL__ !DIR$ OPTIMIZE: 2 #endif }}}

For reference: those comments / pragma are described here: https://software.intel.com/en-us/node/693375#30475674-0480-4EF2-A02E- 68BAD8BBA87F and I checked and indeed our use of the cpp preprocessor hides the {{{!DIR$}}} comment from the Fortran compiler.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:5 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

1:40 p.m.

Comment (by yosef@…):

The code does not seem to be tested by any testsuite. Also, depending on how F90 files are preprocessed, the directive may be hidden. I tried to add Null_Constr output to the existing testsuite, and there are NaNs at t=0 (but not later).

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:6 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

1:50 p.m.

Comment (by Roland Haas):

I checked that our FPP preprocessor line "cpp --traditional" keeps the directive around if {{{__INTEL___}}} is set. I do not know if NaNs at t=0 are of concern or if this is a manifestation of the thorn relying on quantities that are only computed in the evolution loop but not during {{{CCTK_INITIAL}}}.

It not being tested by a testsuite does not terribly surprise me, to tell the truth.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:7 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

1:58 p.m.

Comment (by Yosef Zlochower <yosef@…>):

What sets the __INTEL__ macro?

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:8 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

2:07 p.m.

Comment (by Roland Haas):

The Intel compiler defines this macro (and of course eg gfortran does not) so that one can use it to check if the code is being compiled with an Intel compiler and only disable the optimization in that case, since gfortran does not show the slow compilation.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:9 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

3 Aug 3 Aug

8:05 a.m.

* Attachment "patch_NC" added.

Patch to reduce optimization of Null_Constr code using compiler directives

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

8:05 a.m.

* Attachment "patch_SH" added.

Added Null_Constr output to testsuite

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

8:08 a.m.

Comment (by Yosef Zlochower <yosef@…>):

I attached two patches. One to include the compiler directives, the other the include Null_Constr data in the SphericalHarmonicRecon testsuite. I needed to increase the value of RELTOL to 1.e-9 in order for the latter to pass. I generated the testsuite data from the original NullConstr code first committed to the toolkit (the one that used array operations, rather than explicit loops). The test works against the latest version of NullConstr (with the above mentioned patch)

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:10 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

8:20 a.m.

#2178: PITTNull can take a very long time to compile with the Intel compiler ------------------------------------+--------------------------------- Reporter: Roland Haas | Owner: (none) Type: defect | Status: review Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Resolution: | Keywords: NullConstr ------------------------------------+--------------------------------- Changes (by Roland Haas):

* status: new => review

Comment:

Thank you.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:11 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

8:21 a.m.

#2178: PITTNull can take a very long time to compile with the Intel compiler ------------------------------------+--------------------------------- Reporter: Roland Haas | Owner: (none) Type: defect | Status: review Priority: minor | Milestone: ET_2018_08 Component: EinsteinToolkit thorn | Version: development version Resolution: | Keywords: NullConstr ------------------------------------+--------------------------------- Changes (by Roland Haas):

* milestone: => ET_2018_08

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:12 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

13 Aug 13 Aug

9:25 a.m.

#2178: PITTNull can take a very long time to compile with the Intel compiler ------------------------------------+--------------------------------- Reporter: Roland Haas | Owner: (none) Type: defect | Status: closed Priority: minor | Milestone: ET_2018_08 Component: EinsteinToolkit thorn | Version: development version Resolution: fixed | Keywords: NullConstr ------------------------------------+--------------------------------- Changes (by anonymous):

* status: review => closed * resolution: => fixed

Comment:

Applied as commits:

* 4e86e93 - (HEAD -> master, origin/master, origin/HEAD) SphericalHarmonicRecon: add check for constraints (7 days ago) <Yosef Zlochower> * 6076941 - NullConstr: reduce optimization level to speed up compilation (7 days ago) <Yosef Zlochower> * 2e366e6 - NullConstr: return 0 in first step when constraints cannot be evaluated (7 days ago) <Yosef Zlochower>

of pittnullcode.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/2178#comment:13 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

2979

Age (days ago)

2992

Last active (days ago)

trac@lists.einsteintoolkit.org

15 comments

1 participants

tags (0)

participants (1)

Einstein Toolkit