Comment (by Roland Haas):

I am not quite sure yet if this is working as expected. The SMT thread assignment may be off. I did two test runs one using srun -n 4 --threads-per-core 2 -c 16 and one using srun -n 4 --threads-per-core 1 -c 16 both for a 2 node submission. In both cases I get

This process runs on 16 cores: 0-7, 32-39
Thread 0 runs on 16 cores: 0-7, 32-39

which given the usual logical-cpu to thread mapping are (I think) 16 hardware threads on 8 cores which is not quite what was intended (namely it should have been 16 cores) and leaves some cores empty. In particular the srun -n 4 --threads-per-core 1 -c 16 version should have used only 1 thread per core.

These runs did not use SystemTopology. I will try with it next which may give some more insight.

--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2252/need-for-the-updating-of-the-submit-script