[Users] hwloc: necessary?; explicit dependencies

[Bernard Kelly]

Thanks again, Roland.

For your second point, I've made changes to lib/sbin/MakeUtils.pl (and
also lib/sbin/BuildActiveThorns.pl for the corresponding header
legend), and just submitted them to the ETK ticketing system as an
"enhancement".

Bernard

On Tue, 2 Jul 2019 at 19:06, Haas, Roland <rhaas at illinois.edu> wrote:
>
> Hello Bernard,
>
> > (1) When do I *need* hwloc? Is it only relevant to systems using
> > OpenMPI? The machine Pleiades uses SGI mpi.
> It is very helpful for SystemTopology. SystemTopology in turn makes
> sure that a sane thread-to-core binding is used (most important on
> clusters where hypterthreading is turned on and where the queuing
> system has been "improved" by the admins).
>
> See eg:
>
> https://bitbucket.org/einsteintoolkit/tickets/issues/2252/need-for-the-updating-of-the-submit-script
>
> for such an improved queuing system on Cori where without
> SystemTopology when trying to run on the 16 physical cores (and not
> also the 16 hypterthreading ones) one always gets
>
> This process runs on 16 cores: 0-7, 32-39
> Thread 0 runs on 16 cores: 0-7, 32-39
>
> indicating that the system put 8 threads onto real cores and another 8
> threads onto the corresponding hyperthreads (and did it in such a way
> to have minimal shared use of the L1 cache).
>
> What error do you see? Note that there were a couple of hwloc related
> tickets that may fix your issue:
>
> https://bitbucket.org/einsteintoolkit/tickets/issues/2221/update-hwloc-to-11112
>
> which updates hwloc to 1.11.12 which supports for example KNL. You'd
> get it by updating the hwloc ExternalLibrary to "trunk":
>
> cd arrangements/ExternalLibraries/hwloc
> svn switch  https://github.com/EinsteinToolkit/ExternalLibraries-hwloc.git/trunk
>
> though admittedly due to a bug in the Proca thornlist you likely
> already are on trunk (svn info tells) and can just do "svn up" to get
> the newest version.
>
> Maybe also some others in (though less likely):
>
> https://bitbucket.org/einsteintoolkit/tickets/issues?status=new&status=open&q=hwloc
>
> > (2) More generally, I try to assess module dependencies from what
> > appears in the auto-generated Thornlist. But you can't tell from this
> > that *any* other thorn depends on hwloc. In particular, SystemTopology
> > shows no dependencies at all:
> >
> > CactusUtils/SystemTopology         # SystemTopology ( ) [ ] { }
> >
> > ...  but compilation without hwloc fails immediately unless I comment
> > out SystemTopology as well.
> >
> > Can hidden dependencies like this be improved?
> Probably. That dependency is documented in the configuration.ccl files
> (which is where compilation aborts). My guess would be that the code
> that tries to generate a master thornlist is just not (yet) aware of
> the existence of configuration.ccl file (given that they have only
> existed for 19 years :-P ).
>
> --8<--
> # Configuration definitions for thorn SystemTopology
>
> REQUIRES hwloc MPI
> --8<--
>
> The function to modify would be ThornInfo (last modified 2001-05-07) in
> lib/sbin/MakeUtils.pl which needs to accept "REQUIRES foo bar" and
> "REQUIRES THORNS: foo bar" lines. There's also 'OPTIONAL foo' and
> 'OPTIONAL_IFACTIVE foo'.
>
> If you would like to, please create an enhanced ticket about this and
> there may be someone willing to give it a try (MakeUtils.pl uses *very*
> simplistic perl regex to parse so adding to it is not too hard).
>
> Yours,
> Roland
>
> --
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://keys.gnupg.net.



-- 
------------------------------------------------------------------
Bernard Kelly -- CRESST Assistant Research Scientist, NASA/GSFC
Gravitational Astrophysics Laboratory --- Code 663

Phone: +1 (301) 286-7243 *** Fax: +1 (301) 286-2226
Web: http://science.gsfc.nasa.gov/sed/bio/bernard.j.kelly
ORCID: orcid.org/0000-0002-3326-4454
------------------------------------------------------------------