On Wed, Oct 21, 2015 at 3:49 AM, Roland Haas rhaas@aei.mpg.de wrote:
Hello all,
Glad to hear it's working!
The last line indicates that the threads are not being pinned to specific cores. Usually, we recommend to pin them. You should be able to do this by activating thorns hwloc and SystemTopology in your parameter file.
I suggest to also use your mpirun command's pinning options if available. This can be an issue since MPI will allocate memory before thorns hwloc and SystemTopology have a change of pinning the process so that you may end up with cores having to use memory that is physically attached to the "other" sockets in multi-socket systems.
There is unfortunately nothing any thorn can do about this since MPI may allocate memory as soon as MPI_Init is called by the flesh.
We can introduce a mechanism by which a thorn can be called before MPI_Init is called. This would call all thorns that are compiled in, since the parameter file has not been read yet.
-erik