#2112: move CCTK_MyHost and friends into flesh --------------------------+------------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: Cactus | Version: development version Resolution: | Keywords: Carpet PUGH driver --------------------------+-------------------------------------------------
Comment (by eschnett):
Carpet's implementation is based on comparing host names as strings. This involves significant infrastructure to be able to exchange strings between MPI processes.
On a Blue Gene, the "host name" includes not only what Unix calls host name, but also the core id on which the MPI process is running. This defeats the purpose, and thus one needs to perform string surgery to extract the actual host name from the topology.
OpenMPI provides environment variables that contain the respective information. These are much easier to decode than exchanging host name strings. However, e.g. MPICH does not provide such a mechanism.
Finally, the MPI 3 standard, which is supported by all interesting MPI implementation, but not necessarily by all supercomputers of interest to us, provides a function {{{MPI_Comm_split_type}}}, which accepts an argument {{{MPI_COMM_TYPE_SHARED}}}, which then defines new per-node communicators. With a bit of post-processing, one can extract a relevant grouping: - each per-node communicator chooses a leader (rank 0 within that communicator) - all processes exchange the global rank of their leaders - these ranks can be sorted in ascending order - this ranking then defines an integer numbering of the nodes - you probably want to collect additional statistics in case the number of processes per node isn't uniform