[ET Trac] [Einstein Toolkit] #530: Change default for --num-threads

[Einstein Toolkit]

#530: Change default for --num-threads
--------------------------+-------------------------------------------------
  Reporter:  eschnett     |       Owner:  hinder  
      Type:  enhancement  |      Status:  assigned
  Priority:  major        |   Milestone:          
 Component:  SimFactory   |     Version:          
Resolution:               |    Keywords:          
--------------------------+-------------------------------------------------

Comment (by rhaas):

 Replying to [comment:3 eschnett]:
 > My apologies for missing the discussion. I dislike the idea of
 introducing a global variable for this.
 >
 > If people want to use only a single thread, they can (a) use the --num-
 threads option, or (b) build without OpenMP, or (c) change the settings in
 their defs.local.ini file. A global variable complicates things for
 everybody, and is really just a sign that we do not dare to make a choice
 ourselves. However, this is just what Simfactory is supposed to do -- make
 choices so that people don't have to care. Since with think that OpenMP
 works fine, and since the majority of our code (McLachlan, GRHydro) have
 been parallelised, we should simply make the switch. Or was there somebody
 in the discussion who mentioned an actual problem that would arise?

 I usually turn off OpenMP for my hydro runs since at least for the runs
 that I run (48 cores, lonestar, ideal gas eos) I find that GRHydro with
 OpenMP is about a factor of two slower (!) than without. So even if
 GRHydro might be OpenMP aware and actually contain code to use it, I do
 not run with it since it is not beneficial for me. This might have changed
 by now but was still the case with the development version of GRHydro
 about 2 months or so ago (and I don't think much has changed in GRHydro
 wrt OpenMP since then). So it seems to be there are valid reasons to run
 without OpenMP even if all thorns are not single-threaded.

 As far as fiddling with OMP_NUM_THREADS goes: OpenMP offers a function
 omp_set_num_threads
 (https://computing.llnl.gov/tutorials/openMP/#OMP_SET_NUM_THREADS) to
 change the number of used threads used in parallel sections. It could be
 called by the flesh early on (taking env("OMP_NUM_THREADS") into account).
 This seems better than fooling with the environment variables which should
 be user settable (I think).

 Also it would seem that the information of whether to use or not use
 OpenMP has to be present already at at the time qsub runs since it affects
 the number of MPI processes requested and therefore the flesh alone cannot
 handle it completely it seems to me.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/530#comment:7>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit