[ET Trac] [Einstein Toolkit] #774: hack to unfix OpenMP threads from cores on Cray machines

[Einstein Toolkit]

#774: hack to unfix OpenMP threads from cores on Cray machines
----------------------+-----------------------------------------------------
 Reporter:  rhaas     |       Owner:  eschnett
     Type:  task      |      Status:  new     
 Priority:  optional  |   Milestone:          
Component:  Carpet    |     Version:          
 Keywords:  hack      |  
----------------------+-----------------------------------------------------
 attached is a simple (and not terrible general) patch to have Carpet try
 and spread around the OpenMP threads if the parameter
 cray_thread_affinity_hack is set.

 Right now something like this is required to avoid a 50% slowdown on
 Kraken and Hopper. Do we want something like this in the official sources
 (eventually made a bit more general I'd hope) or is there a way to achieve
 this via say numactl or do we want to keep Carpet clean of hacks for
 individual machines?

 The patch was tried by me and Christian and does indeed unfix the threads.
 It is currently specific a particular arrangement of threads: for "n"
 threads and m MPI processes Kraken seems to affix all the threads from
 process 0 to core 0, all the threads from process 1 to core "n", those of
 process 2 to core "2*n" etc. The patch just makes threads from process "j"
 affine to cores j*n,...,(j-1)*n.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/774>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit