Users May 2013

users@lists.einsteintoolkit.org

13 participants
29 discussions

Different result in buffer zone at iteration 1 only on Gauss
by Yosef Zlochower 23 May '13

23 May '13

Hi, When I tried to run the testsuites on the RIT thorns, the testsuite failed due to an issue at iteration 1, that then disappears. For certain analysis quantities like the Hamiltonian constraint violation and psi4, I find differences in every other bufferzone point between Orsted and Gauss. However, this only occurs at iteration 1 ( my test ran for 16 iterations) My test uses two levels (fixed) of refinement and a relatively coarse grid. On the other hand, I don't see this at all in the adm metric. Where there changes to the prolongation operators that may explain this? My test used 6 buffer zones (there were 4 ghostzones). Here are the relevant Carpet parameters Carpet::domain_from_coordbase = yes Carpet::prolongation_order_space = 5 Carpet::prolongation_order_time = 2 Carpet::max_refinement_levels = 2 Carpet::use_buffer_zones = yes Carpet::num_integrator_substeps = 1 Carpet::additional_buffer_zones = 2 Carpet::verbose = no Carpet::schedule_barriers = no Carpet::veryverbose = no Carpet::init_fill_timelevels = "yes" ActiveThorns="CarpetRegrid2" CarpetRegrid2::regrid_every = 16 CarpetRegrid2::num_centres = 1 CarpetRegrid2::num_levels_1 = 2 CarpetRegrid2::radius_1[ 1] = 5 -- Dr. Yosef Zlochower Center for Computational Relativity and Gravitation Assistant Professor School of Mathematical Sciences Rochester Institute of Technology 85 Lomb Memorial Drive Rochester, NY 14623 Office:74-2067 Phone: +1 585-475-6103 yosef(a)astro.rit.edu CONFIDENTIALITY NOTE: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and destroy any copies of this information.

2 2

New Einstein Toolkit Release
by Frank Loeffler 22 May '13

22 May '13

We are pleased to announce the seventh release (code name "Gauss") of the Einstein Toolkit, an open, community developed software infrastructure for relativistic astrophysics. This release includes various improvements to the MHD handling within GRHydro, and improvements to the Cactus flesh, especially treatment of external libraries. In addition, bug fixes accumulated since the previous release in November 2012 have been included. For more detailed information about the "Gauss" release please read the long release announcement on the Einstein Toolkit web pages: http://einsteintoolkit.org/about/releases/ET_2013_05_announcement.php. The Einstein Toolkit is a collection of software components and tools for simulating and analyzing general relativistic astrophysical systems that builds on numerous software efforts in the numerical relativity community including CactusEinstein, the Carpet AMR infrastructure and the relativistic magneto-hydrodynamics code GRHydro. The Cactus Framework is used as the underlying computational infrastructure providing large-scale parallelization, general computational components, and a model for collaborative, portable code development. The toolkit includes modules to build complete codes for simulating black hole spacetimes as well as systems governed by relativistic magneto-hydrodynamics. The Einstein Toolkit uses a distributed software model and its different modules are developed, distributed, and supported either by the core team of Einstein Toolkit Maintainers, or by individual groups. Where modules are provided by external groups, the Einstein Toolkit Maintainers provide quality control for modules for inclusion in the toolkit and help coordinate support. The Einstein Toolkit Maintainers currently involve postdocs and faculty from six different institutions, and host weekly meetings that are open for anyone to join in. Guiding principles for the design and implementation of the toolkit include: open, community-driven software development; well thought out and stable interfaces; separation of physics software from computational science infrastructure; provision of complete working production code; training and education for a new generation of researchers. For more information about using or contributing to the Einstein Toolkit, or to join the Einstein Toolkit Consortium, please visit our web pages at http://einsteintoolkit.org. The Einstein Toolkit is primarily supported by NSF 1212401/1212426/1212433/1212460 (Einstein Toolkit), 0903973/0903782/0904015 (CIGR), and also by 0905046/0941653 (PetaCactus) and 0710874 (LONI Grid). The Einstein Toolkit contains about 200 regression test cases. On a large portion of the tested machines, almost all of these testsuites pass, using both MPI and OpenMP parallelization. The changes between this and the previous release include: === Compatibility === The following changes lead to potential different numerical results or the need to modify optionlists: * The MPI thorn, as well as several other external libraries, feature an improved autodetection mechanism. Most typical installations don't have to set any options to have installed libraries detected. * The parsing code of some ccl files got re-written. Most existing files should work as they did before, but there are a few corner-cases, e.g. using double quotes around keywords that contain mathematical symbols. === Larger changes since last release === * New components: ** EinsteinAnalysis/Outflow ** Carpet/CycleClock ** LSUThorns/PeriodicCarpet ** CactusTest/TestFortranCrayPointers ** CactusDoc/CoreDoc * simfactory ** updates of various machine configurations ** remove existing default allocations; use local configuration to use these * Cactus flesh: ** Do not buffer output from thorn configuration scripts ** Introduce OPTIONAL_IFACTIVE for thorn dependencies ** Automatically activate required thorns ** Support IBM Blue Gene Q compilers ** Change the parameter parser to Piraha ** Implement CCTK_Error and friends (marked as non-returning for the compiler) ** Replace Cactus complex number type with C/C++ complex numbers ** Workaround for compiler-bug with Intel compiler and 'restrict'. In particular, for the current and recent Intel compilers (version 13), the 'restrict' keyword is disabled in Cactus code by default, because the compiler tends to produce wrong code otherwise. * Carpet ** Also here: too many interesting fixes to mention all, some: ** CarpetIOHDF5: Sliced HDF5 output index file was not stored correctly ** Carpet: Correct load-balancing algorithm to really prefer splitting in the z direction ** CarpetRegrid2: Allow pseudo-2D domains * MoL: Allow skipping MoL's initial copy from past to current timelevel * RotatingSymmetry90: Correct index calculation for "DD" tensors * Formaline: add off-line tarball extraction utility * ADMConstraints: support "new" interface to TmunuBase * AHFinderDirect **Dont use recovered origin if AH was never found **Dont use initial guess origins from parameters when tracking from gridscalar * Hydro_Analysis ** add option to average the location of multiple identical maxima ** add parameters to control which interpolator is used in GRHydro_Separation * HydroBase ** Split temperature and entropy evolution methods ** Add gridfunctions and boilerplate for vector potential * TmunuBase: change the default of "support_old_CalcTmunu_mechanism" to "no" * EOS_Omni ** improve solution for density based on pressure, temperature, Y_e (but slow) ** remove poly_gamma_ini ** add support for cold tabulated EOS (P=P(rho)) that can be supplemented with a thermal gamma law ** add possibility to stich on a polytrope to the lower end of a tabulated cold EOS ** extrapolate to negative temperatures * GRHydro ** remove poly_gamma_ini ** make more paramters steerable ** remove any traces of non-functional Fisheye support ** Introduce variable atmopshere level / tolerance as function of radius. Off by default ** Implemented WENO5 ** Added MP5 reconstruction method ** implemented H viscosity for HLLE solver to eliminate the carbuncle instability ** a lot of smaller fixes/extensions (too many to list here) * GRHydro_InitData ** add "magnetized Bondi" to hydrobase::initial_bvec ** As with GRHydro: a lot of smaller fixes, too many to list here * SetMask_SphericalSurface: fix setup of multiple spheres * HDF5: update to 1.8.10-patch1 * MPI: improved auto-detection, update to OpenMPI 1.6.4 * OpenSSL: update to 1.0.1e * hwloc: library dealing with low-level hardware configurations, version 1.7 * zlib: update to 1.2.8 * Vectors: major update * EinsteinExact: Various, smaller updates (noteworthy, but too many) * Kranc: Various, smaller updates (noteworthy, but too many) * McLachlan ** McLachlan_BSSN: fix the Hamiltonian constraint for CCZ4 ** Fix the sign for 2nd and 6th order dissipation operators === How to upgrade from Ørsted (ET_2012_11) === To upgrade from the previous release, use GetComponents with the new thornlist to check out the new version. See the [http://einsteintoolkit.org/download/ Download] page on the Einstein Toolkit website for download instructions. === Remaining issues with this release === * Certain machines need to be configured specially in Simfactory because the remote directories cannot be determined automatically just from the username. See the Machine notes below. * Recovering with Carpet: The maximum number of timelevels that can be recovered is Carpet::prolongation_order_time+1. This is usually the case, but it is possible to write parameter files e.g. with prolongation_order_time=1 that use 3 timelevels. This bug manifests in an assert() failure when recovering from checkpoints. The work-around is to either increase Carpet::prolongation_order_time or decrease the number of timelevels for the grid function in question accordingly. === Machine notes === * Kraken: defs.local.ini needs to have sourcebasedir = $HOME configured for this machine. You need to determine $HOME by logging in to the machine. * LoneStar and Stampede: defs.local.ini needs to have sourcebasedir = $WORK and basedir = $SCRATCH/simulations configured for this machine. You need to determine $WORK and $SCRATCH by logging in to the machine. * Pandora: The LSU system pandora has at the time of writing not only severe problems of compiling Cactus (can crash the filesystem), but in part due to this also outstanding are some failing testsuite results. Please contact Peter Diener at CCT (LSU) before trying to use this machine. All repositories participating in this release carry a branch ET_2013_05 marking this release. These release branches will be updated if severe errors are found. The "Gauss" Release Team on behalf of the Einstein Toolkit Consortium (2013-05-22) Eloisa Bentivegna Tanja Bode Peter Diener Roland Haas Ian Hinder Frank Löffler Bruno Mundim Christian D. Ott Erik Schnetter May 22, 2013

1 0

New release of the Cactus Computational Toolkit
by Frank Loeffler 22 May '13

22 May '13

We are pleased to announce release 4.2.0 of the Cactus computational toolkit. This release includes support for new architectures, changes to MPI configuration and comes bundled with supporting tools like GetComponents and Simfactory. In addition, bug fixes accumulated since the previous release in November 2012 have been included. This release was tested and released together with the seventh release (code name "Gauss") of the Einstein Toolkit [1], and is used by that. For more detailed information about the 4.2.0 release please read the long release announcement on the Cactus Toolkit web pages: http://cactuscode.org/download/releasenotes/Cactus_4.2.0 On behalf of the Cactus release team, Frank Löffler [1] http://einsteintoolkit.org/about/releases/ET_2013_05_announcement.php

1 0

Carpet: operate on one refinement level?
by Scott Hawley 20 May '13

20 May '13

Hi, I can't seem to get the right combinations of modes correct... I want to do a set of operations on the base grid, i.e. reflevel==0. I've used BEGIN_REFLEVEL_LOOP before, but I no longer want a loop, just level zero. I'm trying this.... BEGIN_GLOBAL_MODE (cctkGH) { BEGIN_LEVEL_MODE (cctkGH) { SWITCH_TO_LEVEL( cctkGH, 0) { //my other code here... } END_SWITCH_TO_LEVEL; } END_LEVEL_MODE; } END_GLOBAL_MODE; ...but I get a failed assertion: Assertion failed: (not is_global_mode()), function level_escape, file /Users/shawley/Cactus/configs/bruiser/build/Carpet/modes.cc, line 1123. If I take out the two lines with "GLOBAL_MODE", I get the same failed assertion. What's the proper way to implement this? Thanks, Scott P.S.- For now, I'll just use... BEGIN_GLOBAL_MODE (cctkGH) { BEGIN_REFLEVEL_LOOP (cctkGH) { if (reflevel != 0) break; //my other code here.... } END_REFLEVEL_LOOP; } END_GLOBAL_MODE; ...but this seems pretty inelegant.

2 1

Meeting Minutes
by Roland Haas 20 May '13

20 May '13

Present: Frank, Steve, Matt, Roland, Yosef, Josh ET release: * most tests pass on all machines, exceptions are GRHydro's weno and slow_sector tests. Roland will look into them (since he put them into the repo). * name of next release: Gauss Yours, Roland

1 0

Einstein Toolkit Meeting Reminder
by Frank Loeffler 20 May '13

20 May '13

Hi Please consider joining the weekly Einstein Toolkit phone call at 10 am US central time on Mondays. As usual, you can find instructions how to join on the following web site: http://einsteintoolkit.org/community/support/ In short: the number is (+1) 225-578-4942 or (+1) 866-573-0359 and the conference id is 118682#. In addition to anything that might be brought up at the meeting, the primary issue at the moment is to get the code ready for a new release next week. Please take a loot at the open bugs flagged as "relevant/interesting" in particular for this release (rather lengthy link on https://trac.einsteintoolkit.org/wiki), currently still 3 (yes: three). Sadly, the last 10 or so only dropped because someone (me) found them to be not as important to hold up the release, so the milestone flag got removed. The three remaining tickets are: 1) Document syntax of new parameter file parser (#1303), Steve? 2) Provide tarball of ET release (#1229), Frank will do 3) CCTK_Info, CCTK_Warn, CCTK_Error not documented (#1267), someone? - ET_2013_05 schedule - May 21: complete freeze, to prepare tar-balls, add tags ect. - May 23: target release date (or earlier if all bugs are fixed) Frank Löffler

1 1

Suggested change in Carpet output file names
by Frank Loeffler 16 May '13

16 May '13

Hi, Several authors debated the plan to change the colons "::" in especially Carpet output file names to something else. One stated reason is the incompatibility of this and Windows, Samba and related systems, another is inconveniences these cause for MacOS users. After considering several options we seem to converge on a replacement of "::" by "--" (double minus signs) as delimiter (see https://trac.einsteintoolkit.org/ticket/1338 for the full discussion). This would, for example change a filename from hydrobase::w_lorentz.maximum.asc to hydrobase--w_lorentz.maximum.asc However small this change is, it would affect the vast majority of users. Thus, we would like to get feedback before actually going forward. So: if you like to have your opinion heard, now is the best time to accomplish that before it is "too late" (whatever direction it might be). Frank

2 1

Grid function "AHFINDERDIRECT::ahmask" has only 1 active time levels on refinement level 1; this is not enough for time interpolation
by Ian Hinder 15 May '13

15 May '13

Hi, I am getting the following error in a BBH simulation just before the merger: > WARNING level 0 in thorn CarpetInterp processor 33 host c438-302.stampede.tacc.utexas.edu > (line 1698 of /work/00915/hinder/Cactus/AEI/arrangements/Carpet/CarpetInterp/src/interp.cc): > -> Grid function "AHFINDERDIRECT::ahmask" has only 1 active time levels on refinement level 1; this is not enough for time interpolation > cactus_sim: /work/00915/hinder/Cactus/AEI/arrangements/Carpet/Carpet/src/helpers.cc:314: int Carpet::Abort(const _cGH *, int): Assertion `0' failed. Does anyone know why this happens, and if there is a workaround I can use to get the run going again? Different resolutions of this run, as well as other runs with similar parameter files, have worked fine. It looks like an edge case. I am using the trunk ET from 16-Apr-2013. Relevant tickets: https://trac.einsteintoolkit.org/ticket/237 (closed) https://trac.einsteintoolkit.org/ticket/380 (still open) I have ADMBase::metric_timelevels set to the default, which is 1. Should it be 3? Would it work to recover with this parameter set to 3? The parameter claims to be steerable, though I don't know what it would use for the previously nonexistent timelevels. How about changing the frequency of AHFinderDirect output to match the coarsest timelevel? -- Ian Hinder http://numrel.aei.mpg.de/people/hinder

3 4

Meeting Minutes
by Roland Haas 13 May '13

13 May '13

Present: Erik, Philipp, Frank, Steve, Tanja, Eloisa, Roland ET release: * most tests pass on all machines, exceptions are GRHydro's weno and slow_sector tests. Roland will look into them (since he put them into the repo). * currently have 16 open tickets flagged for the release. Please have a look. https://trac.einsteintoolkit.org/query?status=%21closed&milestone=ET_2013_0… * release branches were created, please re-run your assigned tests with a release checkout * please apply bug fixes to both trunk and release branch Yours, Roland

1 0

Re: [Users] Einstein Toolkit Meeting Reminder
by Frank Loeffler 12 May '13

12 May '13

Hi Please consider joining the weekly Einstein Toolkit phone call at 10 am US central time on Mondays. As usual, you can find instructions how to join on the following web site: http://einsteintoolkit.org/community/support/ In short: the number is (+1) 225-578-4942 or (+1) 866-573-0359 and the conference id is 118682#. In addition to anything that might be brought up at the meeting, the primary issue at the moment is to get the code ready for a new release in May, with the planned schedule shown below. Please take a loot at the open bugs flagged as "relevant/interesting" in particular for this release (rather lengthy link on https://trac.einsteintoolkit.org/wiki), currently 17. Some of these don't need a lot of attention to get fixed, but 'none' just doesn't work. - ET_2013_05 schedule - May 01: feature freeze of /trunk, extensive testing begins (test-suites on all relevant machines) - May 10: release branches are created trunk unfrozen again release-branch code in only-important-bugs-freeze bug-fixing primarily in release branch - May 21: complete freeze, to prepare tar-balls, add tags ect. - May 23: target release date (or earlier if all bugs are fixed) Frank Löffler

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users May 2013