[ET Trac] [Einstein Toolkit] #2072: Carpet: Disable OpenMP parallelization of transport operators

[Einstein Toolkit]

#2072: Carpet: Disable OpenMP parallelization of transport operators
-------------------------+--------------------------------------------------
 Reporter:  eschnett     |       Owner:  eschnett           
     Type:  enhancement  |      Status:  new                
 Priority:  major        |   Milestone:                     
Component:  Carpet       |     Version:  development version
 Keywords:               |  
-------------------------+--------------------------------------------------
 This disables the OpenMP parallelization of Carpet's transport operators.
 I have observed that this leads to a significant speedup when many threads
 are used.

 The likely reason is that the regions which are parallelized are typically
 small. A typical reason would e.g. be the lower x-boundary of one
 component of one grid variable. The OpenMP thread startup overhead and the
 cache misses caused by parallelizing this are then larger than any
 benefit.

 See <https://bitbucket.org/eschnett/carpet/pull-requests/18/carpet-
 disable-openmp-parallelization-of/diff>.

 In a next step (not proposed here), we can parallelize transport operators
 again, but at a much higher level, e.g. at the level of the loop over all
 variables that need to be prolongated. However, Carpet currently (and
 quite unfortunately) uses static variables to hold pointers to timers, and
 these are not thread-safe. (Neither the static variables nor the timer
 implementation itself are.) This needs to be either corrected or disabled,
 which will be the topic of a further pull request.

 At this time, I ask some of those who are interested in performance in
 trying this pull request on a few iterations of a production simulation
 that uses many OpenMP threads and report back here.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/2072>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit