[Users] meeting minutes for 2018-01-22

[Roland Haas]

Present: Roland, Bhavesh, Nicole Rosato, Miguel, Sam, Yosef, Jim, Erik,
Qian, Carlos, Peter, Antoni Ramos Buades, Zach

OpenMP scalability
* Jim tested different numbers of MPI ranks per node of Stampede2 SKX
* found that 8 ranks per node was best (~10%)
* pure MPI is usually best as long as scaling limit (~1000 MPI ranks)
  has not been reached
* it seems that we are limited by data cache access (at least for
  unigrid where prolongation is not an issue) which is better to
  control with MPI due to separate address spaces which is not what
  happens with OpenMP. Tiling is supposed to help with this, but there
  is no real performance improvement we see.
* recently found the prolongation ops in Carpet use OpenMP in a
  non-ideal way, Erik re-wrote it to parallelize over variables rather
  than over grid points using OpenMP tasks. 
* will be folded back to master branch. Erik's devel branch is
  "eschnett/funhpc", Roland's attempted pull in branch is
  "rhaas/openmp-tasks" which can be used for testing but so far
  comments out all timers. Erik has a suitable new timer class that is
  included in the funhpc branch.
--8<--
in RunScript:
export OMP_STACKSIZE=8000 
in parfile:
# OMP Parallelization
Carpet::use_openmp_in_comm = yes
CarpetLib::use_openmp = no
Carpet::use_openmp_in_sync = yes
Carpet::max_sync_openmp_threads = 0
CarpetLib::enable_openmp_tasks_in_copy = yes
CarpetLib::enable_openmp_tasks_in_comm = yes
CarpetLib::combine_sends = yes
--8<--

Adding Lean public and Proca thorns to the Toolkit
* Lean is a ET evolution code
* Proca is used for scalar (vector actually) fields
* currently publicly available in bitbucket
* needs: http://einsteintoolkit.org/contribute.html
** Components should be of sufficient quality to be used for
peer-reviewed and published science. This includes a basic standard of software engineering, documentation of the software including algorithms and methods, tutorials and examples, and self-tests to demonstrate that the software works correctly.
** To focus development and support, optimize resources, and provide best practises, the Einstein Toolkit components should be of current interest to the community.
** All components must be distributed under an open source licence so that others can use these components without restrictions (except as mandated by scientific integrity), can modify and improve them as necessary, and can pass on these modifications to their collaborators as they see fit.
* it is a bit too late for this release

Release:
* aim for end of February
* current thorns up for inclusion: RNSID
* our rules state that thorns should have been in the development
  thornlist before being in the release. This has not been the case for
  RNSID. Tend to include it anyway since it is a separate thorn and
  there has been honest effort to include it and the failure to have it
  in the thornlist is the maintainers' fault.
* tickets for release:
https://trac.einsteintoolkit.org/query?status=%21closed&milestone=ET_2018_02&or&status=%21closed&priority=critical&component=%21Mojave&or&status=%21closed&priority=blocker&component=%21Mojave&col=id&col=summary&col=milestone&col=priority&col=status&col=owner&col=type&col=component&col=reporter&order=priority
* SphericalHarmonicReconGen:
** could be changes to PITTNull code due to a bugfix
** needs to be actively maintained otherwise it is up for a notice of
removal
** Roland really look at this.
* check on tickets in review state
* have go-through of thorns up for inclusion just after next release,
  add them to thornlist at that point

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20180122/2c4fc3e4/attachment.bin