Users December 2018

users@lists.einsteintoolkit.org

9 participants
13 discussions

Einstein Toolkit Meeting Reminder
by Steven R. Brandt 31 Oct '19

31 Oct '19

Hi Please consider joining the weekly Einstein Toolkit phone call at 9:30 am US central time on Mondays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --Steve

8 94

Agenda for Monday's Meeting
by Steven R. Brandt 30 Oct '19

30 Oct '19

Please update the Wiki with agenda items for Monday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --Steve

3 46

New Einstein Toolkit Release (Chien-Shiung Wu)
by Steven R. Brandt 30 Oct '19

30 Oct '19

=== Release Announcement === We are pleased to announce the sixteenth release (code name "Tesla") of the Einstein Toolkit, an open, community developed software infrastructure for relativistic astrophysics. The highlights of this release are Five new thorns have been added: * GiRaFFE * GiRaFFEfood * ShiftedKerrSchild * GiRaFFE_to_HydroBase * ID_converter_GiRaFFE In addition, bug fixes accumulated since the previous release in Feb 2018 have been included. The Einstein Toolkit is a collection of software components and tools for simulating and analyzing general relativistic astrophysical systems that builds on numerous software efforts in the numerical relativity community including CactusEinstein, the Carpet AMR infrastructure and the relativistic magneto-hydrodynamics code GRHydro. For parts of the toolkit, the Cactus Framework is used as the underlying computational infrastructure providing large-scale parallelization, general computational components, and a model for collaborative, portable code development. The toolkit includes modules to build complete codes for simulating black hole spacetimes as well as systems governed by relativistic magneto-hydrodynamics. The Einstein Toolkit uses a distributed software model and its different modules are developed, distributed, and supported either by the core team of Einstein Toolkit Maintainers, or by individual groups. Where modules are provided by external groups, the Einstein Toolkit Maintainers provide quality control for modules for inclusion in the toolkit and help coordinate support. The Einstein Toolkit Maintainers currently involve postdocs and faculty from six different institutions, and host weekly meetings that are open for anyone to join in. Guiding principles for the design and implementation of the toolkit include: open, community-driven software development; well thought out and stable interfaces; separation of physics software from computational science infrastructure; provision of complete working production code; training and education for a new generation of researchers. For more information about using or contributing to the Einstein Toolkit, or to join the Einstein Toolkit Consortium, please visit our web pages at http://einsteintoolkit.org. The Einstein Toolkit is primarily supported by NSF 1550551/1550461/1550436/1550514 (Einstein Toolkit Community Integration and Data Exploration). The Einstein Toolkit contains about 200 regression test cases. On a large portion of the tested machines, almost all of these tests pass, using both MPI and OpenMP parallelization. The changes between this and the previous release include: == Larger changes since last release == * Support has been added for GCC8. * Built-in OpenMPI tarball has been upgraded to 1.10.7 Note, we still recommend using the system MPI that is installed natively on your system. * And many bug fixes. Notably, a significant correctness problem for NewRad was addressed which caused many simulations with more than two ranks to fail https://trac.einsteintoolkit.org/ticket/2182 == New thorns or tools == * GiRaFFE (~1,600 lines of code): Solves the GRFFE equations in the formalism of Paschalidis and Shapiro [2] using the same reconstruction (PPM) and staggered vector potential methods as IllinoisGRMHD. * GiRaFFEfood (~1,300 lines of code): Code validation initial data and basic diagnostics. Contains Five 1D tests in flat spacetime (fast wave, Alfvén wave, degenerate Alfvén wave, "three-wave", FFE breakdown), One 3D test in flat spacetime (aligned rotator), and Three 3D tests in curved spacetimes ("Exact Wald" solution, "Magnetospheric Wald" solution, and split monopole). * ShiftedKerrSchild (~230 lines of code): Sets up all 3+1 quantities for a Kerr-Schild spacetime with an arbitrary radial shift, to mitigate effects of extreme spacetime curvature near r_{KS}=0. See Appendix of code release paper for full discussion. * GiRaFFE_to_HydroBase and ID_converter_GiRaFFE (~230 lines of code): Acts as a translation layer to convert 3-velocities in HydroBase's Valencia formalism to/from the 3-velocity found in the induction equation (vi = ui/u0; native to GiRaFFE). == Upcoming changes for the next releases == == New thorns planned for the next relases == * Seed_Magnetic_Fields-modified Extended Seed_Magnetic_Fields thorn for binary neutron stars. Supercedes Seed_Magnetic_Fields thorn. * Meudon_Bin_NS-modified Modifications to Meudon BNS initial data thorn to disable the overwriting of initial lapse/shift, which acts to significantly reduce coordinate eccentricity. Supercedes Meudon_Bin_NS thorn. * VolumeIntegrals_GRMHD Nice GRMHD volume integration thorn, currently depends on IllinoisGRMHD and Carpet. Performs volume integrals on arbitrary "Swiss-cheese"-like topologies, and even interoperates with Carpet to track NS centers of mass. * VolumeIntegrals_vacuum Nice GRMHD volume integration thorn, currently depends on ML_BSSN. There is a bit of code duplication and duplicated functionality between VI_GRMHD and VI_vacuum, to ensure that VI_vacuum can be used without enabling a GRMHD code. There is probably a better way of doing this, but I haven't had the time to think deeply about this. * particle_tracerET Solves the ODE \partial_t xi = vi for typically thousands of tracer particles, using an RK4 integration atop the current timestepping. E.g., one RK4 substep in the particle integration might occur every 16 RK4 substeps in the GRMHD evolution. These tracer particle positions are quite useful for visualizing magnetic field lines in a consistent way from frame-to-frame in a movie (recall that in the GRMHD approximation, the magnetic field lines stay attached the fluid elements they thread Flux Freezing?). Note that the velocity must be consistent with the velocity appearing in the GRMHD induction equation. This thorn reads in the HydroBase vel[] vector gridfunction, which assumes the Valencia formalism, and converts it into the induction equation velocity. * smallbPoynET Computes bi, b2, and three spatial components of Poynting flux. It also computes (-1-u0), which is useful for tracking unbound matter. * LeanBSSNMoL and NPScalars: LeanBSSNMoL evolves Einstein's Equations with the BSSN formulation. NPScalars computes the Newman-Penrose scalars. For the conformal factor it uses the "W" version, i.e. W=\gamma^{-1/6}, and it employs the usual "1+log" and "Gamma-driver" gauge conditions. Also available, in the "new_gauge" branch, is the shift condition of Figueras et al for their evolutions of higher dimensional BHs, see: https://arxiv.org/abs/1702.01755 https://arxiv.org/abs/1512.04532 * Proca arrangement: This repository (https://bitbucket.org/canuda/proca/src) provides the tools to evolve the Einstein-Proca system as first described in https://arxiv.org/abs/1505.00797 == How to upgrade from Tesla (ET_2018_02) == To upgrade from the previous release, use GetComponents with the new thornlist to check out the new version. See the Download page (http://einsteintoolkit.org/download/) on the Einstein Toolkit website for download instructions. == Machine notes == Supported (tested) machines include: * Default Debian, Ubuntu, Fedora, CentOS, Mint, OpenSUSE and MacOS (Homebrew and MacPorts) installations * Bluewaters * Comet * Cori * Draco * Edison * Golub * Hydra * Marconi * Minerva * Queenbee 2 * Stampede 2 * SuperMIC * Wheeler * TACC machines: defs.local.ini needs to have sourcebasedir = $WORK and basedir = $SCRATCH/simulations configured for this machine. You need to determine $WORK and $SCRATCH by logging in to the machine. * A new configuration for KNL nodes is being worked on, but not yet included in the release (but compilation works and tests mostly pass). All repositories participating in this release carry a branch ET_2018_02 marking this release. These release branches will be updated if severe errors are found. The "Chien-Shiung Wu" Release Team on behalf of the Einstein Toolkit Consortium (2018-09-17) * Steven R. Brandt * Samuel D. Cupp * Peter Diener * Roland Haas * Roberto De Pietri Sep, 2018

1 1

Re: [Users] [ET Trac] [Einstein Toolkit] #1326: running loopcontrol on strange number of threads fails
by Erik Schnetter 11 Apr '19

11 Apr '19

Do you have a backtrace? -erik On Sunday, March 22, 2015, Einstein Toolkit < trac-noreply(a)einsteintoolkit.org> wrote: > #1326: running loopcontrol on strange number of threads fails > ------------------------------------+--------------------------------------- > Reporter: rhaas | Owner: > Type: defect | Status: new > Priority: minor | Milestone: > Component: EinsteinToolkit thorn | Version: > Resolution: | Keywords: LoopControl > ------------------------------------+--------------------------------------- > > Comment (by rhaas): > > This still happens even with current (Sun Mar 22 18:46:21 CET 2015) trunk, > though failure looks a bit different now: > {{{ > actus_sim: > /data/rhaas/postdoc/gr/ET_trunk/configs/sim/build/LoopControl/loopcontrol.cc:312: > T {anonymous}::divexact(T, T) [with T = int]: Assertion `i % j == 0' > failed > }}} > > -- > Ticket URL: <https://trac.einsteintoolkit.org/ticket/1326#comment:1> > Einstein Toolkit <http://einsteintoolkit.org> > The Einstein Toolkit > _______________________________________________ > Trac mailing list > Trac(a)einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/trac > -- Erik Schnetter <schnetter(a)cct.lsu.edu> http://www.perimeterinstitute.ca/personal/eschnetter/

3 2

Error while compiling ET on cluster
by Jordi Manyer 30 Dec '18

30 Dec '18

Hi, My name is Jordi Manyer from the astrophysics department in Georgia Tech. I'm trying to build the ET on the cluster Cygnus and I've encountered a problem that I think might be related to one of your thorns: After compiling the thorns, Formaline starts creating tarballs and eventually outputs an error: Formaline: Creating configuration git repository... Formaline: Updating files in git repository... Creating Formaline tarball for thorn Carpet Creating Formaline tarball for thorn CarpetIOASCII Creating Formaline tarball for thorn CarpetIOBasic Creating Formaline tarball for thorn CarpetIOScalar Creating Formaline tarball for thorn CarpetInterp Creating Formaline tarball for thorn CarpetReduce Creating Formaline tarball for thorn CarpetRegrid Creating Formaline tarball for thorn CarpetRegrid2 Creating Formaline tarball for thorn CarpetSlab Creating Formaline tarball for thorn WeightedReduce ________________________________________________________________________ Creating cactus_bbh_i15 in /gpfs/pace2/project/pc1/jfuertes7/Cactus/exe from EinsteinBase/ADMBase EinsteinBase/ADMCoupling EinsteinBase/ADMMacros GTThorns/ADM_EJP Numerical/AEILocalInterp EinsteinAnalysis/AHFinderDirect ExternalLibraries/BLAS GTThorns/Baum CactusBase/Boundary Carpet/Carpet Carpet/CarpetIOASCII Carpet/CarpetIOBasic Carpet/CarpetIOScalar Carpet/CarpetInterp Carpet/CarpetLib Carpet/CarpetReduce Carpet/CarpetRegrid Carpet/CarpetRegrid2 Carpet/CarpetSlab Carpet/CarpetTracker CactusBase/CartGrid3D EinsteinBase/Constants CactusBase/CoordBase EinsteinBase/CoordGauge Carpet/CycleClock CactusNumerical/Dissipation EinsteinBase/EOS_Base EinsteinEOS/EOS_IdealFluid CactusElliptic/EllBase CactusElliptic/EllSOR GTThorns/ExternalEtaBeta CactusUtils/Formaline CactusBase/Fortran ExternalLibraries/GSL KrancNumericalTools/GenericFD CactusConnect/HTTPD CactusConnect/HTTPDExtra GTThorns/IHSpin CactusBase/IOASCII CactusBase/IOBasic CactusBase/IOUtil CactusBase/InitBase Kranc/Kranc2BSSNChi ExternalLibraries/LAPACK CactusNumerical/LocalInterp CactusNumerical/LocalReduce Carpet/LoopControl ExternalLibraries/MPI GTThorns/MinTracker CactusNumerical/MoL EinsteinAnalysis/Multipole CactusUtils/NaNChecker CactusUtils/Nice CactusPUGH/PUGH CactusPUGH/PUGHInterp CactusPUGH/PUGHReduce CactusPUGH/PUGHSlab CactusNumerical/Periodic GTThorns/ProperDistance GTThorns/Psi4Analysis EinsteinAnalysis/QuasiLocalMeasures CactusNumerical/ReflectionSymmetry CactusNumerical/RotatingSymmetry180 CactusNumerical/RotatingSymmetry90 GTThorns/RunStats GTThorns/ShiftTracker CactusNumerical/Slab CactusConnect/Socket CactusNumerical/SpaceMask GTThorns/SphereIntegrate GTThorns/SphereRad CactusNumerical/SphericalSurface EinsteinBase/StaticConformal CactusNumerical/SummationByParts CactusBase/SymBase CactusUtils/SystemTopology EinsteinUtils/TGRtensor GTThorns/TeukolskyADMBase CactusBase/Time CactusUtils/TimerReport Carpet/Timers EinsteinBase/TmunuBase CactusUtils/Trigger GTThorns/TwoPunctures CactusUtils/Vectors GTThorns/WaveExtract GTThorns/WeightedReduce EinsteinAnalysis/WeylScal4 GTThorns/Ylm_Decomp ExternalLibraries/hwloc ExternalLibraries/libjpeg ExternalLibraries/zlib fatal: bad revision 'HEAD' Formaline: Committing source tree to git repository... Formaline: Created git tag build-bbh_i15-login-s1.pace.gatech.edu-jfuertes7-2018.11.21-15.14.51-28991 Formaline: Updated git branch config-bbh_i15-login-s1.pace.gatech.edu-gpfs-pace2-project-pc1-jfuertes7-Cactus Formaline: Optimising git repository (slow only the first time)... /gpfs/pace2/project/pc1/jfuertes7/Cactus/configs/bbh_i15/lib/libthorn_Multipole.a(multipole.cc.o): In function `_INTERNAL_85__gpfs_pace2_project_pc1_jfuertes7_Cactus_configs_bbh_i15_build_Multipole_multipole_cc_d1a966a7::output_modes(_cGH*, Multipole::variable_desc const*, double const*, Multipole::mode_array const&)': /gpfs/pace2/project/pc1/jfuertes7/Cactus/configs/bbh_i15/build/Multipole/multipole.cc:128: undefined reference to `Multipole_OutputComplexToH5File(_cGH*, Multipole::variable_desc const*, double const*, Multipole::mode_array const&)' make[1]: *** [/gpfs/pace2/project/pc1/jfuertes7/Cactus/exe/cactus_bbh_i15] Error 1 make[1]: *** Waiting for unfinished jobs.... Formaline: Pushing to master git repository... Formaline: Optimising git repository (slow only the first time)... Formaline: Pushing to local git repository /gpfs/pace2/project/pc1/jfuertes7/Cactus/../CactusSourceJar.git... Formaline: Optimising git repository (slow only the first time)... Formaline: Done. Formaline: (Formaline has finished updating the git repositories. Formaline: If the build process seems to hang, some other command Formaline: is still running, e.g. the final link stage.) make: *** [bbh_i15] Error 2 Have you ever encountered something similar? The compiler in intel 15.0 with mvapich2 2.1 and mkl 11.2 Thanks, Jordi

3 3

meeting minutes for 2018-12-17
by Haas, Roland 30 Dec '18

30 Dec '18

Present: Antonio Ramos Buades, Bill Gabella, Helvi Witek, Peter Diener, Steve Brandt, Roland Haas (late) Names for ET release: * Schwarzschild * Hawking Other items: * Steve went through the page https://docs.einsteintoolkit.org/et-docs/Services identifying updates to info on the infrastructure. * Helvi suggests rotating responsibilities for infrastructure to help get the material more complete. * Roland gave a quick status update on the trac to bitbucet transition project which is in good shape only requiring setting up a translation table to translate user names from trac to bitbucket and some minor fixes to the markdown syntax used (sample with incorrect user names see https://bitbucket.org/rhaas80/et2/issues) * Current ET call time causes conflicts to some participants. Doodle poll for alternative time is here: https://doodle.com/poll/igpv2bvg472nuegt The next ET phone call will be Jan 7th. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

1 0

Einstein Toolkit Meeting Time Doodle Poll
by Steven R. Brandt 21 Dec '18

21 Dec '18

Is there a better time to have the ETK call than Monday morning at 9am CT? Please fill out the following link and let us know what works for you: https://doodle.com/poll/igpv2bvg472nuegt Please check as many times as will work for you. Note that the "If Need Be" option is checked. Thanks. --Steve

1 1

Postdoc in numerical relativity available
by Chris Stevens 17 Dec '18

17 Dec '18

Hi everyone, a two year postdoc is available to further develop code for Cauchy Characteristic Matching in the Cactus framework. Please see the below hyperspace ad for details. https://hyperspace.uni-frankfurt.de/2018/12/14/postdoctoral-position-in-num… Closing date is 16th January. Cheers, Chris -- ru * **Dr Chris Stevens** **Lecturer*: Mathematics Faculty of Science t: +27 (0) 46 603 7547 1 Artillery Road,Grahamstown, 6139*** *PO Box 94, Grahamstown, 6140, South Africa ***www.ru.ac.za* <http://www.ru.ac.za>****www.**c*_*hrisdoesmaths.com*_ <http://www.chrisdoesmaths.com/> *

2 2

meeting minutes for 2018-12-10
by Haas, Roland 10 Dec '18

10 Dec '18

Present: Roland, Antoni, Bill, Helvi, Josh, Miguel, Peter, Sam, Steve, Zach, Helvi, LORENE: * Josh reported on bugs existing in the English named classes in LORENE in the newer part of the codebase that do not yet exist in the French named classed * Josh expects to send bug fixes back to LORENE but will take a while to track them down * not sure if the ET's LORENE1 already has these bugs present that exist in LORENE2 ET infrastructure presentation: * suggestion to move Docker repos etc to github organization * add contact information for ticketing system so that non-locals can ask for help * for LSU try using external LSU accounts created for trac, for NCSA we can create collaborator accounts for maintainers (ask Steve or Roland) * slides are here: https://docs.google.com/presentation/d/1KkEyMEuuTjeW0f4sVRS3cIeIX0XVc3mnQem… * ran out of time, will continue next time Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

Installation problem
by Silvia Traversi 07 Dec '18

07 Dec '18

Hello, I'm trying to install and configure the EinsteinToolkit on my computer (I'm working on Centos). I'm following the tutorial at https://nbviewer.jupyter.org/github/nds-org/jupyter-et/blob/master/CactusTu… . I got the following error during the "Building" phase: /usr/bin/ld: cannot find -lmpi_usempif08 /usr/bin/ld: cannot find -lmpi_usempi_ignore_tkr collect2: error: ld returned 1 exit status make[1]: *** [/home/traversi/Cactus/exe/cactus_sim] Error 1 make: *** [sim] Error 2 Could you help me to fix the problem, please? Thank you very much, Silvia

3 2

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users December 2018