Users May 2019

users@lists.einsteintoolkit.org

11 participants
16 discussions

Einstein Toolkit Meeting Reminder
by Steven R. Brandt 31 Oct '19

31 Oct '19

Hi Please consider joining the weekly Einstein Toolkit phone call at 9:30 am US central time on Mondays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --Steve

8 94

Agenda for Monday's Meeting
by Steven R. Brandt 30 Oct '19

30 Oct '19

Please update the Wiki with agenda items for Monday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --Steve

3 46

New Einstein Toolkit Release (Chien-Shiung Wu)
by Steven R. Brandt 30 Oct '19

30 Oct '19

=== Release Announcement === We are pleased to announce the sixteenth release (code name "Tesla") of the Einstein Toolkit, an open, community developed software infrastructure for relativistic astrophysics. The highlights of this release are Five new thorns have been added: * GiRaFFE * GiRaFFEfood * ShiftedKerrSchild * GiRaFFE_to_HydroBase * ID_converter_GiRaFFE In addition, bug fixes accumulated since the previous release in Feb 2018 have been included. The Einstein Toolkit is a collection of software components and tools for simulating and analyzing general relativistic astrophysical systems that builds on numerous software efforts in the numerical relativity community including CactusEinstein, the Carpet AMR infrastructure and the relativistic magneto-hydrodynamics code GRHydro. For parts of the toolkit, the Cactus Framework is used as the underlying computational infrastructure providing large-scale parallelization, general computational components, and a model for collaborative, portable code development. The toolkit includes modules to build complete codes for simulating black hole spacetimes as well as systems governed by relativistic magneto-hydrodynamics. The Einstein Toolkit uses a distributed software model and its different modules are developed, distributed, and supported either by the core team of Einstein Toolkit Maintainers, or by individual groups. Where modules are provided by external groups, the Einstein Toolkit Maintainers provide quality control for modules for inclusion in the toolkit and help coordinate support. The Einstein Toolkit Maintainers currently involve postdocs and faculty from six different institutions, and host weekly meetings that are open for anyone to join in. Guiding principles for the design and implementation of the toolkit include: open, community-driven software development; well thought out and stable interfaces; separation of physics software from computational science infrastructure; provision of complete working production code; training and education for a new generation of researchers. For more information about using or contributing to the Einstein Toolkit, or to join the Einstein Toolkit Consortium, please visit our web pages at http://einsteintoolkit.org. The Einstein Toolkit is primarily supported by NSF 1550551/1550461/1550436/1550514 (Einstein Toolkit Community Integration and Data Exploration). The Einstein Toolkit contains about 200 regression test cases. On a large portion of the tested machines, almost all of these tests pass, using both MPI and OpenMP parallelization. The changes between this and the previous release include: == Larger changes since last release == * Support has been added for GCC8. * Built-in OpenMPI tarball has been upgraded to 1.10.7 Note, we still recommend using the system MPI that is installed natively on your system. * And many bug fixes. Notably, a significant correctness problem for NewRad was addressed which caused many simulations with more than two ranks to fail https://trac.einsteintoolkit.org/ticket/2182 == New thorns or tools == * GiRaFFE (~1,600 lines of code): Solves the GRFFE equations in the formalism of Paschalidis and Shapiro [2] using the same reconstruction (PPM) and staggered vector potential methods as IllinoisGRMHD. * GiRaFFEfood (~1,300 lines of code): Code validation initial data and basic diagnostics. Contains Five 1D tests in flat spacetime (fast wave, Alfvén wave, degenerate Alfvén wave, "three-wave", FFE breakdown), One 3D test in flat spacetime (aligned rotator), and Three 3D tests in curved spacetimes ("Exact Wald" solution, "Magnetospheric Wald" solution, and split monopole). * ShiftedKerrSchild (~230 lines of code): Sets up all 3+1 quantities for a Kerr-Schild spacetime with an arbitrary radial shift, to mitigate effects of extreme spacetime curvature near r_{KS}=0. See Appendix of code release paper for full discussion. * GiRaFFE_to_HydroBase and ID_converter_GiRaFFE (~230 lines of code): Acts as a translation layer to convert 3-velocities in HydroBase's Valencia formalism to/from the 3-velocity found in the induction equation (vi = ui/u0; native to GiRaFFE). == Upcoming changes for the next releases == == New thorns planned for the next relases == * Seed_Magnetic_Fields-modified Extended Seed_Magnetic_Fields thorn for binary neutron stars. Supercedes Seed_Magnetic_Fields thorn. * Meudon_Bin_NS-modified Modifications to Meudon BNS initial data thorn to disable the overwriting of initial lapse/shift, which acts to significantly reduce coordinate eccentricity. Supercedes Meudon_Bin_NS thorn. * VolumeIntegrals_GRMHD Nice GRMHD volume integration thorn, currently depends on IllinoisGRMHD and Carpet. Performs volume integrals on arbitrary "Swiss-cheese"-like topologies, and even interoperates with Carpet to track NS centers of mass. * VolumeIntegrals_vacuum Nice GRMHD volume integration thorn, currently depends on ML_BSSN. There is a bit of code duplication and duplicated functionality between VI_GRMHD and VI_vacuum, to ensure that VI_vacuum can be used without enabling a GRMHD code. There is probably a better way of doing this, but I haven't had the time to think deeply about this. * particle_tracerET Solves the ODE \partial_t xi = vi for typically thousands of tracer particles, using an RK4 integration atop the current timestepping. E.g., one RK4 substep in the particle integration might occur every 16 RK4 substeps in the GRMHD evolution. These tracer particle positions are quite useful for visualizing magnetic field lines in a consistent way from frame-to-frame in a movie (recall that in the GRMHD approximation, the magnetic field lines stay attached the fluid elements they thread Flux Freezing?). Note that the velocity must be consistent with the velocity appearing in the GRMHD induction equation. This thorn reads in the HydroBase vel[] vector gridfunction, which assumes the Valencia formalism, and converts it into the induction equation velocity. * smallbPoynET Computes bi, b2, and three spatial components of Poynting flux. It also computes (-1-u0), which is useful for tracking unbound matter. * LeanBSSNMoL and NPScalars: LeanBSSNMoL evolves Einstein's Equations with the BSSN formulation. NPScalars computes the Newman-Penrose scalars. For the conformal factor it uses the "W" version, i.e. W=\gamma^{-1/6}, and it employs the usual "1+log" and "Gamma-driver" gauge conditions. Also available, in the "new_gauge" branch, is the shift condition of Figueras et al for their evolutions of higher dimensional BHs, see: https://arxiv.org/abs/1702.01755 https://arxiv.org/abs/1512.04532 * Proca arrangement: This repository (https://bitbucket.org/canuda/proca/src) provides the tools to evolve the Einstein-Proca system as first described in https://arxiv.org/abs/1505.00797 == How to upgrade from Tesla (ET_2018_02) == To upgrade from the previous release, use GetComponents with the new thornlist to check out the new version. See the Download page (http://einsteintoolkit.org/download/) on the Einstein Toolkit website for download instructions. == Machine notes == Supported (tested) machines include: * Default Debian, Ubuntu, Fedora, CentOS, Mint, OpenSUSE and MacOS (Homebrew and MacPorts) installations * Bluewaters * Comet * Cori * Draco * Edison * Golub * Hydra * Marconi * Minerva * Queenbee 2 * Stampede 2 * SuperMIC * Wheeler * TACC machines: defs.local.ini needs to have sourcebasedir = $WORK and basedir = $SCRATCH/simulations configured for this machine. You need to determine $WORK and $SCRATCH by logging in to the machine. * A new configuration for KNL nodes is being worked on, but not yet included in the release (but compilation works and tests mostly pass). All repositories participating in this release carry a branch ET_2018_02 marking this release. These release branches will be updated if severe errors are found. The "Chien-Shiung Wu" Release Team on behalf of the Einstein Toolkit Consortium (2018-09-17) * Steven R. Brandt * Samuel D. Cupp * Peter Diener * Roland Haas * Roberto De Pietri Sep, 2018

1 1

ToV Collapse Example Parfile
by COLE KELSON-PACKER 19 Jun '19

19 Jun '19

Hello, I am a new user of the einstein toolkit, and I've been going through running some of the example par files provided by default. I've been trying to test the tov collapse par file tov_collapse_vc_13.par. Looking around online, I've found that it runs using a thorn called CoreCollapseControl not included in the current build. I've found slightly modified versions in the archives of this mailing list that run using the thorn Trigger, but these don't seem to work either. I am aware that this simulation eventually returns GRHydro errors, but the par file isn't even fully compiling for me. Everything seems to build smoothly until the following error: "Warning[L1,P0] (Cactus): Major error in parameter file 'output-0000/tov_collapse_vc_13.par line 74: Parameter 'loopcontrol::use_random_restart_hill_climbing' not found" this is followed by warnings coming from scheduling and routine, then the simulation aborts. This suggests to me that something has changed in the loopcontrol thorn since, but I don't know. The parfile I'm trying to use come from the conversation in lists.einsteintoolkit.org/pipermail/users/2012-October/002592.html I would be grateful if anyone could shed light on what's going on here. Thank you! -- Cole Kelson-Packer Department of Physics, University of Utah

3 2

interpolation error
by 林家暉 30 May '19

30 May '19

Hello, I am using IllinoisGRMHD with the test with nsns_test.par in https://bitbucket.org/zach_etienne/wvuthorns_diagnostics/src/master/NSNS_pa… . The simulation runs well until t~5300 and an error occurs. The error message is like this: WARNING level 1 from host nid02210 process 0 while executing schedule bin particle_tracerET, routine particle_tracerET::DoOneRK4StepForParticleTracerET in thorn AEILocalInterp, file /global/u1/c/chlin/et/Cactus/arrangements/Numerical/AEILocalInterp/src/Lagrange-tensor-product/../template.c:1109: -> CCTK_InterpLocalUniform(): interpolation point is either outside the grid, or inside but too close to the grid boundary! (this may be caused by a global interpolation with driver::ghost_size too small) 0-origin interpolation point number pt=0 of N_interp_points=1 interpolation point (x,y,z)=(-157.725,-576.349,-99.2027) grid x_min(delta_x)x_max = -576(18)576 grid y_min(delta_y)y_max = -576(18)54 grid z_min(delta_z)z_max = -576(18)54 WARNING level 1 from host nid02210 process 0 while executing schedule bin particle_tracerET, routine particle_tracerET::DoOneRK4StepForParticleTracerET in thorn particle_tracerET, file /global/u1/c/chlin/et/Cactus/arrangements/WVUThorns_Diagnostics/particle_tracerET/src/Interpolate_velocities_at_particle_positions.C:131: -> interpolation screwed up WARNING level 1 from host nid02210 process 1 while executing schedule bin particle_tracerET, routine particle_tracerET::DoOneRK4StepForParticleTracerET in thorn particle_tracerET, file /global/u1/c/chlin/et/Cactus/arrangements/WVUThorns_Diagnostics/particle_tracerET/src/Interpolate_velocities_at_particle_positions.C:131: -> interpolation screwed up srun: error: nid02211: tasks 2-3: Exited with exit code 1 srun: Terminating job step 21753671.0 srun: error: nid02210: tasks 0-1: Exited with exit code 1 I think maybe t can be avoided by decreasing the timestep so that the tracer particle would not move too fast to make the interpolation screwed up but I am not sure. So what I want to ask is : 1. It this error can be avoided by decreasing the timestep ? 2. If so, how can I decrease the timestep in parameter file (or in other places)? Thank you. Best regards, Chia-Hui

3 5

Installing the toolkit
by Κοτοπούλη ς Δημήτρης 30 May '19

30 May '19

Dear sir or madam, I am currently thinking of turning to Linux so that using the Einstein Toolkit becomes easier. Which Linux version should I prefer? Thanks for your time.

3 2

meeting minutes for 2019-05-30
by Haas, Roland 30 May '19

30 May '19

Present: Bill, Helvi, Peter, Roland, Sam, Toni, Steve, Yosef, Zach proposed changes to Einstein Toolkit citing policy: * will decide on July 12th * currently the ET as a whole is cited via the original paper Loeffler et al., however the author list has become stale wrt to currently involved people * newer software toolkits have transitioned to cite the software rather than a paper. This makes it easier for new people to contribute since they know they will get credit, helping software sustainability and supporting careers in software * there were no objections to this idea from the call participants Peter's self-force code into the ET: * code is not quite ready to include yet * is currently working on documentation * is currently working on adding a Cactus like test infrastructure * would go towards making the ET a true multi-code project * could be important as well for future funding of the ET * important for LISA science LISA science goals: * needs some IMRI capable code * (almost) equal mass needs much higher accuracy since it needs to be removed as a background from data * currently the LISA internal discussion is dominated by one code (which is not the ET) gallery examples: * TOV has been updated to match the ET paper * working on having a downsized version of this for the tutorial: all in agreement, will proceed this way * BBH example is in progress ET meeting at RIT: * PLEASE register * schedule is being finalized due to last minute changes * will broadcast over bluejeans for remote participation Peter reported in a very recent paper on a bssn like formulation of bimetric gravity. He contacted the authors about their interest in using the ET, which they are and had already looked at the ET. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

meeting minutes for 2019-05-23
by Haas, Roland 23 May '19

23 May '19

Present: Roland, Zach, Antoni, Chris, Sam, Steve, Yosef proposed updates to citation policy: * will send email with details to mailing list trac to bitbucket switch: * Steve removed all mention of trac from website * Steve set up redirects on former trac website to bitbucket tickets Jenkins server down: * had break in to server, which was then used for bitcoin mining * Ian is working on restoring from backups * Roland to get a bit more involved to be able to do basic maintenance New ET components * Fishbone-Moncrief disk should be almost ready for inclusion ** has analysis code written around that, that could be used for a full science simulation ** postprocessing is mix of thorns and python code, was used for Event-Horizon-Telescope code comparison project Using the ET a library * Zach has a proof of concept hook into the ET that lets one use the ET as a library and inject data into it eg for horizon finding * useful to reach out to other projects * repo is here: https://bitbucket.org/zach_etienne/et_minimal_2019_03 which overwrites some Cactus functions in the ETK_interface.c file. The remainder code is vanilla Cactus. LISA meeting: * Steve and Sam attended the LISA meeting * have needs for very high accuracy and accurate phase * so far Cactus does not seem to be on their map * Intermediate (q=100) mass ratio inspiral (IMRI) runs are the most interesting ones, but not really something the ET can easily do. Some results exists by the RIT group. * Zach is working with students to link some PN work (EOB mostly) with ET and numerical relativity, this can be used to study 4PN effects RIT meeting: * register soon to enable head count Wiki to bitbucket wiki: * Steve will provide XML dump * running is currently not so much the issue, but the current LDAP plugin is no longer maintained and forces us to run specific versions PreSync: * can run all tests with PreSync turned off * no other updates * Sam, Steve and Roland to come up with a plan, so that this can be reported on in the report to NSF * Sam will present on this in PEARC July 28th to August 1st ET working groups: * need to take stock of where they are * need to restart presentations and discussions Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

Bug in Piraha / trac.einsteintoolkit.org is down
by Erik Schnetter 21 May '19

21 May '19

I just found and corrected a problem in how Piraha handles the "min" and "max" functions in parameter files: (1) Piraha processes the first argument twice. (This isn't really a bug since min and max are idempotent.) (2) Piraha stops after processing the first element since a "return" statement is placed wrong. The diff below corrects both problems. -erik $ git diff diff --git a/src/piraha/Call.cc b/src/piraha/Call.cc index cf777d58..facd2565 100644 --- a/src/piraha/Call.cc +++ b/src/piraha/Call.cc @@ -405,7 +405,7 @@ smart_ptr<Value> meval(smart_ptr<Group> gr,ExpressionEvaluationData *eedata) { std::string par = get_parfile(); CCTK_Error(gr->line(),par.c_str(),current_thorn.c_str(),msg.str().c_str()); } - for (int i=1; i<=gr->groupCount(); i++) { + for (int i=2; i<=gr->groupCount(); i++) { smart_ptr<Value> val_next = meval(gr->group(i),eedata); // Make sure all arguments are either integer or real if (val_next->type != PIR_REAL && val_next->type != PIR_INT) { @@ -437,8 +437,8 @@ smart_ptr<Value> meval(smart_ptr<Group> gr,ExpressionEvaluationData *eedata) { std::string par = get_parfile(); CCTK_Error(gr->line(),par.c_str(),current_thorn.c_str(),"internal error"); } - return val; } + return val; } // From here on only functions that take exactly one argument: the majority. else if (gr->groupCount() != 2) { -- Erik Schnetter <schnetter(a)gmail.com> http://www.perimeterinstitute.ca/personal/eschnetter/

3 2

Jenkins build and test system
by Ian Hinder 20 May '19

20 May '19

Hi, Unfortunately, I have reason to believe that the ET Jenkins build and test system has again been compromised by cryptocurrency mining malware, so it has been taken offline. We have backups from before the intrusion, but need to identify the security flaw which allowed this before restoring the last good backup, changing all security tokens, and bringing the system back online. -- Ian Hinder Research Software Engineer University of Manchester, UK

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users May 2019