Users February 2019

users@lists.einsteintoolkit.org

10 participants
18 discussions

Einstein Toolkit Meeting Reminder
by Steven R. Brandt 31 Oct '19

31 Oct '19

Hi Please consider joining the weekly Einstein Toolkit phone call at 9:30 am US central time on Mondays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --Steve

8 94

Agenda for Monday's Meeting
by Steven R. Brandt 30 Oct '19

30 Oct '19

Please update the Wiki with agenda items for Monday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --Steve

3 46

New Einstein Toolkit Release (Chien-Shiung Wu)
by Steven R. Brandt 30 Oct '19

30 Oct '19

=== Release Announcement === We are pleased to announce the sixteenth release (code name "Tesla") of the Einstein Toolkit, an open, community developed software infrastructure for relativistic astrophysics. The highlights of this release are Five new thorns have been added: * GiRaFFE * GiRaFFEfood * ShiftedKerrSchild * GiRaFFE_to_HydroBase * ID_converter_GiRaFFE In addition, bug fixes accumulated since the previous release in Feb 2018 have been included. The Einstein Toolkit is a collection of software components and tools for simulating and analyzing general relativistic astrophysical systems that builds on numerous software efforts in the numerical relativity community including CactusEinstein, the Carpet AMR infrastructure and the relativistic magneto-hydrodynamics code GRHydro. For parts of the toolkit, the Cactus Framework is used as the underlying computational infrastructure providing large-scale parallelization, general computational components, and a model for collaborative, portable code development. The toolkit includes modules to build complete codes for simulating black hole spacetimes as well as systems governed by relativistic magneto-hydrodynamics. The Einstein Toolkit uses a distributed software model and its different modules are developed, distributed, and supported either by the core team of Einstein Toolkit Maintainers, or by individual groups. Where modules are provided by external groups, the Einstein Toolkit Maintainers provide quality control for modules for inclusion in the toolkit and help coordinate support. The Einstein Toolkit Maintainers currently involve postdocs and faculty from six different institutions, and host weekly meetings that are open for anyone to join in. Guiding principles for the design and implementation of the toolkit include: open, community-driven software development; well thought out and stable interfaces; separation of physics software from computational science infrastructure; provision of complete working production code; training and education for a new generation of researchers. For more information about using or contributing to the Einstein Toolkit, or to join the Einstein Toolkit Consortium, please visit our web pages at http://einsteintoolkit.org. The Einstein Toolkit is primarily supported by NSF 1550551/1550461/1550436/1550514 (Einstein Toolkit Community Integration and Data Exploration). The Einstein Toolkit contains about 200 regression test cases. On a large portion of the tested machines, almost all of these tests pass, using both MPI and OpenMP parallelization. The changes between this and the previous release include: == Larger changes since last release == * Support has been added for GCC8. * Built-in OpenMPI tarball has been upgraded to 1.10.7 Note, we still recommend using the system MPI that is installed natively on your system. * And many bug fixes. Notably, a significant correctness problem for NewRad was addressed which caused many simulations with more than two ranks to fail https://trac.einsteintoolkit.org/ticket/2182 == New thorns or tools == * GiRaFFE (~1,600 lines of code): Solves the GRFFE equations in the formalism of Paschalidis and Shapiro [2] using the same reconstruction (PPM) and staggered vector potential methods as IllinoisGRMHD. * GiRaFFEfood (~1,300 lines of code): Code validation initial data and basic diagnostics. Contains Five 1D tests in flat spacetime (fast wave, Alfvén wave, degenerate Alfvén wave, "three-wave", FFE breakdown), One 3D test in flat spacetime (aligned rotator), and Three 3D tests in curved spacetimes ("Exact Wald" solution, "Magnetospheric Wald" solution, and split monopole). * ShiftedKerrSchild (~230 lines of code): Sets up all 3+1 quantities for a Kerr-Schild spacetime with an arbitrary radial shift, to mitigate effects of extreme spacetime curvature near r_{KS}=0. See Appendix of code release paper for full discussion. * GiRaFFE_to_HydroBase and ID_converter_GiRaFFE (~230 lines of code): Acts as a translation layer to convert 3-velocities in HydroBase's Valencia formalism to/from the 3-velocity found in the induction equation (vi = ui/u0; native to GiRaFFE). == Upcoming changes for the next releases == == New thorns planned for the next relases == * Seed_Magnetic_Fields-modified Extended Seed_Magnetic_Fields thorn for binary neutron stars. Supercedes Seed_Magnetic_Fields thorn. * Meudon_Bin_NS-modified Modifications to Meudon BNS initial data thorn to disable the overwriting of initial lapse/shift, which acts to significantly reduce coordinate eccentricity. Supercedes Meudon_Bin_NS thorn. * VolumeIntegrals_GRMHD Nice GRMHD volume integration thorn, currently depends on IllinoisGRMHD and Carpet. Performs volume integrals on arbitrary "Swiss-cheese"-like topologies, and even interoperates with Carpet to track NS centers of mass. * VolumeIntegrals_vacuum Nice GRMHD volume integration thorn, currently depends on ML_BSSN. There is a bit of code duplication and duplicated functionality between VI_GRMHD and VI_vacuum, to ensure that VI_vacuum can be used without enabling a GRMHD code. There is probably a better way of doing this, but I haven't had the time to think deeply about this. * particle_tracerET Solves the ODE \partial_t xi = vi for typically thousands of tracer particles, using an RK4 integration atop the current timestepping. E.g., one RK4 substep in the particle integration might occur every 16 RK4 substeps in the GRMHD evolution. These tracer particle positions are quite useful for visualizing magnetic field lines in a consistent way from frame-to-frame in a movie (recall that in the GRMHD approximation, the magnetic field lines stay attached the fluid elements they thread Flux Freezing?). Note that the velocity must be consistent with the velocity appearing in the GRMHD induction equation. This thorn reads in the HydroBase vel[] vector gridfunction, which assumes the Valencia formalism, and converts it into the induction equation velocity. * smallbPoynET Computes bi, b2, and three spatial components of Poynting flux. It also computes (-1-u0), which is useful for tracking unbound matter. * LeanBSSNMoL and NPScalars: LeanBSSNMoL evolves Einstein's Equations with the BSSN formulation. NPScalars computes the Newman-Penrose scalars. For the conformal factor it uses the "W" version, i.e. W=\gamma^{-1/6}, and it employs the usual "1+log" and "Gamma-driver" gauge conditions. Also available, in the "new_gauge" branch, is the shift condition of Figueras et al for their evolutions of higher dimensional BHs, see: https://arxiv.org/abs/1702.01755 https://arxiv.org/abs/1512.04532 * Proca arrangement: This repository (https://bitbucket.org/canuda/proca/src) provides the tools to evolve the Einstein-Proca system as first described in https://arxiv.org/abs/1505.00797 == How to upgrade from Tesla (ET_2018_02) == To upgrade from the previous release, use GetComponents with the new thornlist to check out the new version. See the Download page (http://einsteintoolkit.org/download/) on the Einstein Toolkit website for download instructions. == Machine notes == Supported (tested) machines include: * Default Debian, Ubuntu, Fedora, CentOS, Mint, OpenSUSE and MacOS (Homebrew and MacPorts) installations * Bluewaters * Comet * Cori * Draco * Edison * Golub * Hydra * Marconi * Minerva * Queenbee 2 * Stampede 2 * SuperMIC * Wheeler * TACC machines: defs.local.ini needs to have sourcebasedir = $WORK and basedir = $SCRATCH/simulations configured for this machine. You need to determine $WORK and $SCRATCH by logging in to the machine. * A new configuration for KNL nodes is being worked on, but not yet included in the release (but compilation works and tests mostly pass). All repositories participating in this release carry a branch ET_2018_02 marking this release. These release branches will be updated if severe errors are found. The "Chien-Shiung Wu" Release Team on behalf of the Einstein Toolkit Consortium (2018-09-17) * Steven R. Brandt * Samuel D. Cupp * Peter Diener * Roland Haas * Roberto De Pietri Sep, 2018

1 1

Re: [Users] [ET Trac] [Einstein Toolkit] #1326: running loopcontrol on strange number of threads fails
by Erik Schnetter 11 Apr '19

11 Apr '19

Do you have a backtrace? -erik On Sunday, March 22, 2015, Einstein Toolkit < trac-noreply(a)einsteintoolkit.org> wrote: > #1326: running loopcontrol on strange number of threads fails > ------------------------------------+--------------------------------------- > Reporter: rhaas | Owner: > Type: defect | Status: new > Priority: minor | Milestone: > Component: EinsteinToolkit thorn | Version: > Resolution: | Keywords: LoopControl > ------------------------------------+--------------------------------------- > > Comment (by rhaas): > > This still happens even with current (Sun Mar 22 18:46:21 CET 2015) trunk, > though failure looks a bit different now: > {{{ > actus_sim: > /data/rhaas/postdoc/gr/ET_trunk/configs/sim/build/LoopControl/loopcontrol.cc:312: > T {anonymous}::divexact(T, T) [with T = int]: Assertion `i % j == 0' > failed > }}} > > -- > Ticket URL: <https://trac.einsteintoolkit.org/ticket/1326#comment:1> > Einstein Toolkit <http://einsteintoolkit.org> > The Einstein Toolkit > _______________________________________________ > Trac mailing list > Trac(a)einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/trac > -- Erik Schnetter <schnetter(a)cct.lsu.edu> http://www.perimeterinstitute.ca/personal/eschnetter/

3 2

meeting minutes for 2019-02-28
by Haas, Roland 28 Feb '19

28 Feb '19

Present: Roland, Chris, Helvi, Yosef, Peter, Sam, Bill, Steve, Antoni, Zach ET release * BNS gallery example - no updates * WVU_Diagnostics - 2 out of 5 thorns have tests now - added all READMEs including usage examples - Zach will add the remaining test cases - Jenkins tests currently running: https://build.barrywardell.net/ * Proca / Lean - no updates - Peter will look at https://bitbucket.org/einsteintoolkit/tickets/issues/2101/add-lean_public-a… to verify the changes that Helvi and Roland made LISA symposium: * discussed that it would be good if a number of ET representatives would be at the symposium http://lists.einsteintoolkit.org/pipermail/users/2019-February/006775.html . Currently Helvi is the only person known to attend. * the next consortium meeting on April 29th, in Florida Gainesville, https://lisacon4.phys.ufl.edu/ ET US meeting: * invited speakers list is about to be finalized * email asking for registration is about to be sent out ET tutorial server: * signup page for accounts still missing * 404 page on server still missing Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

Fwd: LISA Consortium Meeting #4
by Witek, Helvi 28 Feb '19

28 Feb '19

Dear all, here is the official announcement for next LISA Consortium meeting in Gainesville, Florida. Best wishes, Helvi =========================================== Dr. Helvi Witek Royal Society University Research Fellow Theoretical Particle Physics and Cosmology Department of Physics King's College London =========================================== ---------- Forwarded message --------- From: Mueller,Guido <gum1(a)ufl.edu<mailto:gum1@ufl.edu>> Date: Thu, Feb 28, 2019 at 3:01 PM Subject: LISA Consortium Meeting #4 To: consortium(a)lisamission.org<mailto:consortium@lisamission.org> <consortium(a)lisamission.org<mailto:consortium@lisamission.org>>, martin.gehler(a)esa.int<mailto:martin.gehler@esa.int> <martin.gehler(a)esa.int<mailto:martin.gehler@esa.int>>, Paul McNamara <pmcnamar(a)cosmos.esa.int<mailto:pmcnamar@cosmos.esa.int>>, Ira Thorpe <james.i.thorpe(a)nasa.gov<mailto:james.i.thorpe@nasa.gov>>, Livas, Jeffrey (GSFC-6630) <jeffrey.livas-1(a)nasa.gov<mailto:jeffrey.livas-1@nasa.gov>>, Kelly Holley-Bockelmann <k.holley(a)vanderbilt.edu<mailto:k.holley@vanderbilt.edu>> Cc: Conklin,John <jwconklin(a)ufl.edu<mailto:jwconklin@ufl.edu>>, Wass,Peter <pwass(a)ufl.edu<mailto:pwass@ufl.edu>>, Fulda, Paul <pfulda(a)phys.ufl.edu<mailto:pfulda@phys.ufl.edu>>, Simon Barke <barke(a)ufl.edu<mailto:barke@ufl.edu>>, LISA Conference Admin <lisacon(a)phys.ufl.edu<mailto:lisacon@phys.ufl.edu>> Dear all, The LISA Consortium meeting #4 will be held at the University of Florida in Gainesville, Florida. The meeting proper will start on Monday, April 29th and ends on Wednesday, May 1st. Pre- and post meetings are also possible for smaller groups; we are currently working on the schedule. We plan to have a public presentation on Sunday 6pm prior to the meeting followed by a reception. Details are still TBD. Registration is now possible on our meeting website: https://lisacon4.phys.ufl.edu<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flisacon4…> NOTE: The LISA Consortium meeting is not an open meeting and access is restricted to LISA Consortium members and people working directly on the LISA study or LISA project at the space agencies or at our industrial partners. So please do not distribute this link widely but inform the people who need to know. We will work with Martin Hewitson to approve all registrations. Ira, Jeff, Paul, Martin: Please distribute the link among your LISA team members at the space agencies and your industrial partners at your discretion. All of them are certainly more than welcome. But I don’t have a good list to reach them. The registration fee will be paid through eventbrite, a platform the University has chosen to handle financial transactions that involve third party credit cards (your credit cards). Just in case you are irritated by that. Best regards Paul, Peter, John, Simon, and me Guido Mueller Professor of Physics University of Florida

1 0

Change of job and email address
by Ian Hinder 27 Feb '19

27 Feb '19

Hi all, Sorry for the long absence from the list! In November, I started a new job at University of Manchester where I am now employed as a Research Software Engineer, working on a variety of research computing projects across the university, mostly focused on HPC. I will try to keep up with the ET a little, though obviously not as much as before. Please update your address books! ian.hinder(a)manchester.ac.uk<mailto:ian.hinder@manchester.ac.uk>. My AEI email address will stop working at the end of February. -- Ian Hinder Research Software Engineer University of Manchester, UK

1 0

First LISA Waveform Working Group Workshop
by Witek, Helvi 26 Feb '19

26 Feb '19

Dear all, the first stand-alone meeting of the LISA Waveform working group (WavWG) will take place at the AEI Potsdam from 13 - 15 May 2019. The WavWG is dedicated to the development of waveform templates for the LISA mission and connects the broader scientific community to the LISA Consortium. As such, the WavWG coordinates efforts to model a broad spectrum of gravitational wave sources, establishing links with LISA’s astrophysics, cosmology, fundamental physics and simulation working groups, and bridge between communities employing different methodologies to prepare waveforms. This meeting is the first stand-alone meeting of the WavWG and aims at getting together leading international experts and young scientists to identify pressing tasks concerning LISA waveform modelling in its broadest sense and to foster new international collaborations. The meeting will have few talks per day and provide ample time for discussions. While the WavWG is part of the LISA Consortium, you do not need to be in the Consortium to attend. You can find additional information here:. https://workshops.aei.mpg.de/lisawavwg/ https://hyperspace.uni-frankfurt.de/2019/02/20/lisa-waveform-working-group-… The registration will end on *30 April 2019*. We have limited funds available to support PhD students. Should you require financial support, please follow the instructions on our website and apply until *30 March 2019*. Best wishes, Deirdre Shoemaker, Maarten van de Meent, Niels Warburton, Helvi Witek (Co-chairs LISA Waveform Working Group) =========================================== Dr. Helvi Witek Royal Society University Research Fellow Theoretical Particle Physics and Cosmology Department of Physics King's College London ===========================================

1 0

meeting minutes for 2019-02-21
by Haas, Roland 21 Feb '19

21 Feb '19

Present: Roland, Steve, Sam, Bill, Chris, Zach, Helvi, Antoni, Future of svn.cactuscode.org [SB] - suggest to check access log - only thing hosted there that we know to be still in use is website source for cactuscode.org and CarpetHDF5 visit reader - repositories should / must be moved to some other hosting service before ET release planning [RH] * release team - Steve - Roland - Helvi - Peter - Sam * release goals ** BNS gallery example - no update ** WVU_Analysis / WVU_Diagnostics status - Zach has been adding missing pieces to thorn - Add thorns to master thornlist of manifest repo - Tests are still missing, Zach is trying to obtain tiny ID dataset, most thorns could use more generic data though - Zach is looking for a place to put example parfiles ** Proca / Lean status - Lean has example parfiles that could go to Cactus/par - thorns are in thornlist - removed SIMD vectorization which caused issues with Intel 17 / 18 compilers on Stampede2. See https://bitbucket.org/einsteintoolkit/tickets/issues/2101/ ** release relevant tickets https://bitbucket.org/einsteintoolkit/tickets/issues?milestone=ET_2019_02&m… - sqrt() issue on AVX512 https://bitbucket.org/einsteintoolkit/tickets/issues/2219/fix-sqrt-for-avx5… should be fixed. The pull request contains more than just the bugfix. Bugfix commit is here: https://bitbucket.org/cactuscode/cactusutils/commits/5dea378d702207d771e644… Stale example parfiles * suggestion to at least run the example parfiles through cactus_sim -S to test that the parfile are at least parsable * could be made part of the jenkins tests Workshop at RIT - no updates unanswered question on mailing list / open tickets sorted by update time - no unanswered question on mailing list Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

2 2

Meudon_Mag_NS gives segmentation fault
by Lorenzo Sala 19 Feb '19

19 Feb '19

Dear EinsteinToolkit users, I am having some trouble in using the thorn Meudon_Mag_NS. In order to run the parameter file "bbig.par" (in the par directory), I have to remove every reference to the thorn ADMConstraints and, moreover, the thorn itself is activated as ID_Mag_NS instead of Meudon_Mag_NS. However, even after fixing the errors in the parameter file, I'm given back a segmentation fault, as you can see in the attached file. If you have any suggestions, please let me know. Thanking you in advance, Lorenzo Sala

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users February 2019