[ET Trac] [Einstein Toolkit] #2112: move CCTK_MyHost and friends into flesh
Einstein Toolkit
trac-noreply at einsteintoolkit.org
Wed Feb 7 12:19:39 CST 2018
#2112: move CCTK_MyHost and friends into flesh
--------------------------+-------------------------------------------------
Reporter: rhaas | Owner:
Type: enhancement | Status: new
Priority: minor | Milestone:
Component: Cactus | Version: development version
Resolution: | Keywords: Carpet PUGH driver
--------------------------+-------------------------------------------------
Comment (by eschnett):
Carpet's implementation is based on comparing host names as strings. This
involves significant infrastructure to be able to exchange strings between
MPI processes.
On a Blue Gene, the "host name" includes not only what Unix calls host
name, but also the core id on which the MPI process is running. This
defeats the purpose, and thus one needs to perform string surgery to
extract the actual host name from the topology.
OpenMPI provides environment variables that contain the respective
information. These are much easier to decode than exchanging host name
strings. However, e.g. MPICH does not provide such a mechanism.
Finally, the MPI 3 standard, which is supported by all interesting MPI
implementation, but not necessarily by all supercomputers of interest to
us, provides a function {{{MPI_Comm_split_type}}}, which accepts an
argument {{{MPI_COMM_TYPE_SHARED}}}, which then defines new per-node
communicators. With a bit of post-processing, one can extract a relevant
grouping:
- each per-node communicator chooses a leader (rank 0 within that
communicator)
- all processes exchange the global rank of their leaders
- these ranks can be sorted in ascending order
- this ranking then defines an integer numbering of the nodes
- you probably want to collect additional statistics in case the number of
processes per node isn't uniform
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/2112#comment:1>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
More information about the Trac
mailing list