[Users] simfactory's num-smt and max-num-smt options and hyperthreading

[Roland Haas]

Hello Erik,

thank you for the quick response.

> The file "processterminology.rst" in Simfactory is supposed to define
> these. I see that this file is incomplete -- it does not describe pus, nor
> does it describe the Simfactory machine database keys. The fact that pus
> are missing isn't that bad, since no queuing system I know of can allocate
> individual pus anyway. Also, using hyperthreads looks from a software side
> the same way as oversubscribing cores.
I see. Thank you for the explanations. I had forgotten about the file.
I think I disagree with the "same way as oversubscribing cores."
statement (not the idea, just the wording) since it seems as if the OS
on stampede2 does report 96 processors (in /proc/cpuinfo's terminology)
so a naive application will assume that they are all identical. This is
mincing words though :-).

> The various misconfigurations and misunderstandings are supposed to be
> "translated away" by Simfactory. Thus Stampede2's SKX nodes have 48 cores
> in Simfactory, and if the queueing system requires specifying this as 96,
> then Simfactory needs to multiply these numbers by 2 when calling sbatch or
> srun.
ok.

> I would try with this description of Stampede2's SKX nodes:
> 
> max-num-smt     = 2   # the system has hyperthreading
> num-smt         = 1   # ignore hyperthreading by default
> ppn             = 48   # physical cores
> mpn             = 2   # NUMA domains ("sockets")
mpn? I guess is makes sense. Initially I would have used "spn", which I
realize is really the number of physical CPU sockets but each socket
could have multiple NUMA domains. Note that neither "mpn" or "spn" is
used by simfactory for anything (ie "git grep [sm]pn" does only find
them in ini files). One of the ini files being
etc/syntax/mdb-syntax.ini which states for mpn "memory banks per node"
which is a bit ambiguous (could be NUMA domains or could be physical
memory sockets) and for spn gives "sockets per node" which is not that
much better.

> max-num-threads = 96   # using more threads will oversubscribe the pus
How odd, I would have expected this to fail (max-num-treads > ppn) but
simfactory happily accepts it and "--procs 96 --num-threads 96
--num-smt 2" gives me a single node while "--procs 96 --num-threads 96 --num-smt 1" requests 2 nodes, which what I wanted.

> num-threads     = 24   # should probably change this
Yup, this is now 2 in simfactory's master.

I will add these settings to rhaas/stampede2 to see if Jim wants to
give that branch a try as well.

Thank you for the explanations.

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20180221/958f123f/attachment.bin