Users January 2018

users@lists.einsteintoolkit.org

17 participants
31 discussions

meeting minutes for 2018-01-22
by Roland Haas 22 Jan '18

22 Jan '18

Present: Roland, Bhavesh, Nicole Rosato, Miguel, Sam, Yosef, Jim, Erik, Qian, Carlos, Peter, Antoni Ramos Buades, Zach OpenMP scalability * Jim tested different numbers of MPI ranks per node of Stampede2 SKX * found that 8 ranks per node was best (~10%) * pure MPI is usually best as long as scaling limit (~1000 MPI ranks) has not been reached * it seems that we are limited by data cache access (at least for unigrid where prolongation is not an issue) which is better to control with MPI due to separate address spaces which is not what happens with OpenMP. Tiling is supposed to help with this, but there is no real performance improvement we see. * recently found the prolongation ops in Carpet use OpenMP in a non-ideal way, Erik re-wrote it to parallelize over variables rather than over grid points using OpenMP tasks. * will be folded back to master branch. Erik's devel branch is "eschnett/funhpc", Roland's attempted pull in branch is "rhaas/openmp-tasks" which can be used for testing but so far comments out all timers. Erik has a suitable new timer class that is included in the funhpc branch. --8<-- in RunScript: export OMP_STACKSIZE=8000 in parfile: # OMP Parallelization Carpet::use_openmp_in_comm = yes CarpetLib::use_openmp = no Carpet::use_openmp_in_sync = yes Carpet::max_sync_openmp_threads = 0 CarpetLib::enable_openmp_tasks_in_copy = yes CarpetLib::enable_openmp_tasks_in_comm = yes CarpetLib::combine_sends = yes --8<-- Adding Lean public and Proca thorns to the Toolkit * Lean is a ET evolution code * Proca is used for scalar (vector actually) fields * currently publicly available in bitbucket * needs: http://einsteintoolkit.org/contribute.html ** Components should be of sufficient quality to be used for peer-reviewed and published science. This includes a basic standard of software engineering, documentation of the software including algorithms and methods, tutorials and examples, and self-tests to demonstrate that the software works correctly. ** To focus development and support, optimize resources, and provide best practises, the Einstein Toolkit components should be of current interest to the community. ** All components must be distributed under an open source licence so that others can use these components without restrictions (except as mandated by scientific integrity), can modify and improve them as necessary, and can pass on these modifications to their collaborators as they see fit. * it is a bit too late for this release Release: * aim for end of February * current thorns up for inclusion: RNSID * our rules state that thorns should have been in the development thornlist before being in the release. This has not been the case for RNSID. Tend to include it anyway since it is a separate thorn and there has been honest effort to include it and the failure to have it in the thornlist is the maintainers' fault. * tickets for release: https://trac.einsteintoolkit.org/query?status=%21closed&milestone=ET_2018_0… * SphericalHarmonicReconGen: ** could be changes to PITTNull code due to a bugfix ** needs to be actively maintained otherwise it is up for a notice of removal ** Roland really look at this. * check on tickets in review state * have go-through of thorns up for inclusion just after next release, add them to thornlist at that point Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

New time for the Einstein Toolkit calls: 7:00am PST / 9:00am CST / 10:00am EST / 15:00 CET
by Roland Haas 22 Jan '18

22 Jan '18

Hello all, as pointed out in the ET reminder email, the new meeting time is 9:00 am US central time on Mondays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

2 1

Using Stampede2 SKX
by James Healy 20 Jan '18

20 Jan '18

Hello all, I am trying to run on the new skylake processors on Stampede2 and while the run speeds we are obtaining are very good, we are concerned that we aren't optimizing properly when it comes to OpenMP. For instance, we see the best speeds when we use 8 MPI processors per node (with 6 threads each for a total of 48 total threads/node). Based on the architecture, we were expecting to see the best speeds with 2 MPI/node. Here is what I have tried: 1. Using the simfactory files for stampede2-skx (config file, run and submit scripts, and modules loaded) I compiled a version of ET_2017_06 using LazEv (RIT's evolution thorn) and McLachlan and submitted a series of runs that change both the number of nodes used, and how I distribute the 48 threads/node between MPI processes. 2. I use a standard low resolution grid, with no IO or regridding. Parameter file attached. 3. Run speeds are measured from Carpet::physical_time_per_hour at iteration 256. 4. I tried both with and without hwloc/SystemTopology. 5. For both McLachlan and LazEv, I see similar results, with 2 MPI/node giving the worst results (see attached plot for McLachlan) and a slight preferences for 8 MPI/node. So my questions are: 1. Has there been any tests run by any other users on stampede2 skx? 2. Should we expect 2 MPI/node to be the optimal choice? 3. If so, are there any other configurations we can try that could help optimize? Thanks in advance! Jim Healy

2 1

Release plans
by ian.hinder＠aei.mpg.de 20 Jan '18

20 Jan '18

Hi all, As we discussed in the call last Monday, it would be good to finalise the list of tickets that must be addressed for the upcoming release. If you have time, please look through the open tickets and add the ET_2018_02 milestone to those that we should consider. We can then decide which ones we have time to work on for this release, and which ones will be bumped to the next release. Thanks! -- Ian Hinder http://members.aei.mpg.de/ianhin

1 0

meeting minutes for 2018-01-15
by Roland Haas 15 Jan '18

15 Jan '18

Present: Steve, Roland, Miguel Garcia, Antoni Ramos Buades, Yosef, Bill, Ian Change of time for call: * as a result of the doodle poll the time of the call has changed to 9:00am CST / 16:00 CET ET release: * preliminary aim for release in week of Feb 21st * release steps are outlined in https://docs.einsteintoolkit.org/et-docs/Release_Process * thorns to include: ** RNSID ** Giraffe * Roland mentions continuing issues with ML constraints, they are in ticket: https://trac.einsteintoolkit.org/ticket/1995 need to understand what is going on US ET meeting: * final decisions about date are being doing done, looking for rooms right now * schedule is being decided * no more details at this point * so far only workshop, since EU one will have a school * Bill and Steve mentioned that the lack of school was seens a bad thing for the Mallorca one Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

1 0

Meeting Time
by Steven R. Brandt 10 Jan '18

10 Jan '18

Here is the doodle poll for the ET call time. Note that the meetings are 1 hour, despite what the poll says, and that the poll isn't just about next week, but about establishing a (possibly) new regular time. Thanks. --Steve

1 1

F90 in GRHydro
by Cupp, Samuel 09 Jan '18

09 Jan '18

Hey all, I am working on implementing the new synchronization and boundary condition application methods for the GRHydro thorn. I missed today's call, but I see that the Fortran code is being removed in the new release. Are those changes on a branch in the remote repository? I would like to merge those changes into my branch as soon as I can. Alternatively, is there a rough timeline by which I can expect these changes to be merged with the master branch? Samuel Cupp

2 1

meeting minutes for 2018-01-08
by Roland Haas 08 Jan '18

08 Jan '18

Present: Steve, Roland, Bhavesh, Andres Mauricio Suarez Mantilla, Antoni Ramos Buades, Bill, Ian Release planning: * plan is to release in February * wiki has some pages explaining the process: https://docs.einsteintoolkit.org/et-docs/Main_Page#Release_planning * have at least: ** RNSID thorn to include ** Giraffe thorn to be included ** remove F90 code from GRHydro ** move ExternalLibraries to github * Steve to find out how to create release milestones * Steve to find out how to delegate trac permissions * review machines to test on in particular KNL machines * send announcement about release planning to the mailing list Jenkins server: * still down * Ian will follow up ET call time: * come up with proposed time for call * would prefer whole hours since it simplifies scheduling US ET workshop: * second week of May for now, will know for sure by end of this week Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

WaveDemo Errors
by Andres Mauricio Suarez Mantilla 08 Jan '18

08 Jan '18

Hi Everybody. I just installed Cactus and einsteintoolkit.th ran without any problem. However, I tried to run WaveDemo thorn and it showed some problems (please see the attached image). I really don't know how to fix such errors, so I hope you can help me with this. I appreciate your support. Pdta: I am working in LinuxMint 18.1 Serena. Best regards, Mauricio Suárez Departamento de Física Universidad Nacional de Colombia

2 1

WaveDemo - Error
by Andres Mauricio Suarez Mantilla 06 Jan '18

06 Jan '18

HI everybody I just installed Cactus on my laptop (LinuxMint 18.1 Serena) but I had a problem running WaveDemo thorn: ----------------------------------------------------- CST error 1: -> Configuration script for thorn PETSC returned exit code 2 Error message: 'There is no support for external PETSc installations for the PETSc architecture 'arch-linux2-c-debug'. Please set the variable PETSC_ARCH_LIBS manually, and/or have Cactus build PETSc, and/or send a request to <cactusmaint(a)cactuscode.org>.' ---------------------------------------------------------- I tried to export path to PETSc installation directory but it did not work. Can somebody help me please?, I appreciate your attention and support. Best regards Mauricio Suárez Departamento de Física Universidad Nacional de Colombia

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users January 2018