[Users] hwloc: necessary?; explicit dependencies

[Haas, Roland]

Hello Bernard,

> (1) When do I *need* hwloc? Is it only relevant to systems using
> OpenMPI? The machine Pleiades uses SGI mpi.
It is very helpful for SystemTopology. SystemTopology in turn makes
sure that a sane thread-to-core binding is used (most important on
clusters where hypterthreading is turned on and where the queuing
system has been "improved" by the admins).

See eg:

https://bitbucket.org/einsteintoolkit/tickets/issues/2252/need-for-the-updating-of-the-submit-script

for such an improved queuing system on Cori where without
SystemTopology when trying to run on the 16 physical cores (and not
also the 16 hypterthreading ones) one always gets

This process runs on 16 cores: 0-7, 32-39
Thread 0 runs on 16 cores: 0-7, 32-39

indicating that the system put 8 threads onto real cores and another 8
threads onto the corresponding hyperthreads (and did it in such a way
to have minimal shared use of the L1 cache).

What error do you see? Note that there were a couple of hwloc related
tickets that may fix your issue:

https://bitbucket.org/einsteintoolkit/tickets/issues/2221/update-hwloc-to-11112

which updates hwloc to 1.11.12 which supports for example KNL. You'd
get it by updating the hwloc ExternalLibrary to "trunk":

cd arrangements/ExternalLibraries/hwloc
svn switch  https://github.com/EinsteinToolkit/ExternalLibraries-hwloc.git/trunk

though admittedly due to a bug in the Proca thornlist you likely
already are on trunk (svn info tells) and can just do "svn up" to get
the newest version.

Maybe also some others in (though less likely):

https://bitbucket.org/einsteintoolkit/tickets/issues?status=new&status=open&q=hwloc

> (2) More generally, I try to assess module dependencies from what
> appears in the auto-generated Thornlist. But you can't tell from this
> that *any* other thorn depends on hwloc. In particular, SystemTopology
> shows no dependencies at all:
> 
> CactusUtils/SystemTopology         # SystemTopology ( ) [ ] { }
> 
> ...  but compilation without hwloc fails immediately unless I comment
> out SystemTopology as well.
> 
> Can hidden dependencies like this be improved?
Probably. That dependency is documented in the configuration.ccl files
(which is where compilation aborts). My guess would be that the code
that tries to generate a master thornlist is just not (yet) aware of
the existence of configuration.ccl file (given that they have only
existed for 19 years :-P ).

--8<--
# Configuration definitions for thorn SystemTopology

REQUIRES hwloc MPI
--8<--

The function to modify would be ThornInfo (last modified 2001-05-07) in
lib/sbin/MakeUtils.pl which needs to accept "REQUIRES foo bar" and
"REQUIRES THORNS: foo bar" lines. There's also 'OPTIONAL foo' and
'OPTIONAL_IFACTIVE foo'.

If you would like to, please create an enhanced ticket about this and
there may be someone willing to give it a try (MakeUtils.pl uses *very*
simplistic perl regex to parse so adding to it is not too hard).

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20190702/5f19bc31/attachment-0001.bin