Users

users@lists.einsteintoolkit.org

3 participants
3984 discussions

New Einstein Toolkit Release (Brahe)
by Frank Loeffler 16 Jun '16

16 Jun '16

We are pleased to announce the thirteenth release (code name "Brahe") of the Einstein Toolkit, an open, community developed software infrastructure for relativistic astrophysics. This release includes, among other things, updates to the IllinoisGRMHD code, updated requirements concerning C++, and updates examples in the gallery. In addition, bug fixes accumulated since the previous release in November 2015 have been included. For more detailed information about the "Brahe" release please read the long release announcement on the Einstein Toolkit web pages: http://einsteintoolkit.org/about/releases/ET_2016_05_announcement.php. The Einstein Toolkit is a collection of software components and tools for simulating and analyzing general relativistic astrophysical systems that builds on numerous software efforts in the numerical relativity community including CactusEinstein, the Carpet AMR infrastructure and the relativistic magneto-hydrodynamics code GRHydro. For parts of the toolkit, the Cactus Framework is used as the underlying computational infrastructure providing large-scale parallelization, general computational components, and a model for collaborative, portable code development. The toolkit includes modules to build complete codes for simulating black hole spacetimes as well as systems governed by relativistic magneto-hydrodynamics. The Einstein Toolkit uses a distributed software model and its different modules are developed, distributed, and supported either by the core team of Einstein Toolkit Maintainers, or by individual groups. Where modules are provided by external groups, the Einstein Toolkit Maintainers provide quality control for modules for inclusion in the toolkit and help coordinate support. The Einstein Toolkit Maintainers currently involve postdocs and faculty from six different institutions, and host weekly meetings that are open for anyone to join in. Guiding principles for the design and implementation of the toolkit include: open, community-driven software development; well thought out and stable interfaces; separation of physics software from computational science infrastructure; provision of complete working production code; training and education for a new generation of researchers. For more information about using or contributing to the Einstein Toolkit, or to join the Einstein Toolkit Consortium, please visit our web pages at http://einsteintoolkit.org. The Einstein Toolkit is primarily supported by NSF 1212401/1212426/1212433/1212460 (Einstein Toolkit), and also by 0905046/0941653 (PetaCactus) and 0710874 (LONI Grid). The Einstein Toolkit contains about 200 regression test cases. On a large portion of the tested machines, almost all of these test suites pass, using both MPI and OpenMP parallelization. The changes between this and the previous release include: === Larger changes since last release === * Cactus Flesh - Enable C++11 by default, and require it to be supported Support bibtex in LaTeX thorn documentation External libraries can specify additional paths (only) added as include path for Fortran. - The 'checkout' target of 'make' has been removed, as it only supported cvs, was unused and untested for a long time. * IllinoisGRMHD/Convert_to_HydroBase: major update * SetMask_SphericalSurface: modify criterion for setting the mask to avoid problems with intermittent apparent horizon finder failures More examples in the Einstein Toolkit Gallery (http://einsteintoolkit.org/about/gallery/) * Simfactory - New configurations: acrcluster, centos, cori, minerva, sunnyvale - Updates to several machines (too many to list) - stampede: fix core affinity settings * New thorns or tools - New thorn WatchDog: a thorn that terminates jobs that do not make progress over a user-defined time frame, defined as time in between ANALYSIS steps. === How to upgrade from Somerville (ET_2015_11) === To upgrade from the previous release, use GetComponents with the new thornlist to check out the new version. Switching to the new branch is not possible for most repositories due to repository moves. See the [http://einsteintoolkit.org/download Download] page on the Einstein Toolkit website for download instructions. === Remaining issues with this release === * Certain machines need to be configured specially in Simfactory because the remote directories cannot be determined automatically just from the user name. See the Machine notes below. * Recovering with Carpet: Carpet stores metadata (such as the simulation time) only for Carpet::max_timelevels time levels, although it is possible to allocate more time levels. These additional time levels then cannot be recovered; the symptom is an assertion failure during recovery. The solution is to either increase Carpet::max_timelevels, or to decrease the number of active time levels. === Machine notes === Supported (tested) machines include: - Default Debian, Ubuntu, Fedora, CentOS, and MacOS installations - Bethe - Bluewaters - Comet - Datura - Fionn - Gordon - GPC - Gullimin - MP2 - Nvidia - Queenbee 2 - Shelob - Supermike II - Stampede (CPU) - Titan - Zwicky * Stampede: defs.local.ini needs to have sourcebasedir = $WORK and basedir = $SCRATCH/simulations configured for this machine. You need to determine $WORK and $SCRATCH by logging in to the machine. All repositories participating in this release carry a branch ET_2016_05 marking this release. These release branches will be updated if severe errors are found. The "Brahe" Release Team on behalf of the Einstein Toolkit Consortium (2016-06-15) Steven R. Brandt Peter Diener Roland Haas Ian Hinder Frank Löffler Bruno C. Mundim Erik Schnetter Barry Wardell June 16, 2016

1 0

Final testing of upcoming release
by Frank Loeffler 14 Jun '16

14 Jun '16

Hi, Sorry about the re-post. I should have send this the first time with a better Subject and in it's own thread: > Hi > > On Tue, Jun 14, 2016 at 08:06:23AM +0200, Frank Loeffler wrote: > > > I plan to create release branches in about 8 hours (or so). > > The release branches have been created, and you can use the following > URL to get a checkout of it (remember, GetComponents also takes URLs): > > https://bitbucket.org/einsteintoolkit/manifest/raw/ET_2016_05/einsteintoolk… > > This does NOT mean the release it out of the door yet. > > If we don't see any errors, this will be what will be released, > hopefully soon. The more errors we can detect now the better. Critical > things would still be fixed, non-critical can be mentioned in the > release notes. In order for any of this to happen, we need > > * YOU * > > to test the to-be release. > > Frank

2 3

Re: [Users] [Einstein Toolkit] #1910: ExternalLibraries/HDF5 does not set HDF5_INC_DIRS
by Frank Loeffler 14 Jun '16

14 Jun '16

On Mon, Jun 13, 2016 at 09:36:59AM -0000, Einstein Toolkit wrote: > #1910: ExternalLibraries/HDF5 does not set HDF5_INC_DIRS > ----------------------------+----------------------------------------------- > Reporter: barry.wardell | Owner: eschnett > Type: defect | Status: assigned > Priority: unset | Milestone: > Component: Other | Version: development version > Resolution: | Keywords: > ----------------------------+----------------------------------------------- > > Comment (by knarf): > > Looking at this, HDF5_INC_DIRS should be set by bash_utils.sh (assuming > you set HDF5_DIR, but none of the other variables in your option list). > > Can you please attach details about your setup, in particular: > - How is your hdf5 library installed (which INC_DIRS, LIB_DIRS, LIBS would > be correct)? > - Your option list > - The output of detect.sh, when called by Cactus as part of a build I plan to create release branches in about 8 hours (or so). It would be splendid if this could be fixed until then. If not, it will need to be fixed on both trunk and the release branch, until tomorrow at about the time of this email. If it isn't 'in' until then, it will not be part of the initial release, but would be a prime candidate for a fix later. Current plan for action is release on Wednesday. I will not treat this as blocker for the reason that this is unlikely to affect most users, and the ones that are affected are not unlikely to be experienced enough to find a workaround. Frank

1 2

Instructions for setting up the ET
by Ian Hinder 14 Jun '16

14 Jun '16

Hi, I have expanded the page https://docs.einsteintoolkit.org/et-docs/Compiling_the_Einstein_Toolkit to be a general page on how to compile the ET, including the different cases of i) A supported cluster; ii) A supported or unsupported OS on a laptop or workstation; and iii) An unsupported cluster. The latter is of course incomplete since is it complex. There are obvious places in which people could add more information. Some of the information on https://docs.einsteintoolkit.org/et-docs/Simplified_Tutorial_for_New_Users is now duplicated; we could use a MediaWiki template page and include it on both pages, or we could link to the compilation page from the tutorial page. The main duplication is the table of optionlists etc for supported workstation/laptop operating systems. This could actually be removed completely if "sim setup" knew how to choose these. There are many unsatisfactory aspects to the whole process, and it is likely to be intimidating for new users. It would be good to streamline this by actually thinking about a new user who is trying to do all of this for the first time, and seeing if simfactory or the instructions could make things easier. Please take a look and feel free to improve it. -- Ian Hinder http://members.aei.mpg.de/ianhin

2 1

Jenkins offline
by Barry Wardell 14 Jun '16

14 Jun '16

Hi, The machine that runs the Jenkins unit tests is currently offline. I am trying to get it back online but it is difficult at the weekend, so it may be Monday before it is back. Barry

2 2

Re: [Users] [Commits] [svn:einsteintoolkit] www/about/gallery/gw150914/ (Rev. 1559)
by Frank Loeffler 13 Jun '16

13 Jun '16

On Mon, Jun 13, 2016 at 02:38:29PM +0000, www-push(a)einsteintoolkit.org wrote: > User: www-push > Date: 2016/06/13 09:38 AM > > Added: > /about/gallery/gw150914/ > compile_and_run.php > > Log: > gw150914: Add compile_and_run.php It would be nice not to use the www-push user for non-automatic commits like this - simply to know the author. Frank

2 1

No Einstein Toolkit Meeting
by Frank Loeffler 12 Jun '16

12 Jun '16

Hi, There will *not* be an Einstein Toolkit meeting today (Mon Feb 08th). The next meeting will be on Mon Feb 15th, 2016, at the new time (half an hour earlier than 'usually': 9:30am US central time). Frank Loeffler

1 1

Re: [Users] Users Digest, Vol 75, Issue 10
by David Bradley 11 Jun '16

11 Jun '16

On 06/11/2016 12:00 PM, users-request(a)einsteintoolkit.org wrote: > Send Users mailing list submissions to > users(a)einsteintoolkit.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.einsteintoolkit.org/mailman/listinfo/users > or, via email, send a message with subject or body 'help' to/usr/lib/x86_64-linux-gnu > users-request(a)einsteintoolkit.org > > You can reach the person managing the list at > users-owner(a)einsteintoolkit.org > > When replying, please edit your Subject line so it is more > than "Re: Contents of Users digest..." > > > I also think relying on local libraries may be a good thing. Newbies may not know how to locate the correct libraries and include files for the Linux systems. I have been in a position of finding some distributions put libraries in unusual locations. In fact the following may be a problem for some: /usr/lib/x86_64-linux-gnu/libhdf5.a (Linux Mint 17.3) It may be expected in /usr/lib tree, base tree. Feedback of locations in other systems ( distributions) may be help in this matter. David

1 0

Thoughts on external libraries
by Erik Schnetter 10 Jun '16

10 Jun '16

The Einstein Toolkit is a complex piece of software. Almost by definition, and certainly by design, the Cactus framework chooses to build (and thus depend) upon other software, rather than re-inventing the wheel. Thus we have to deal with linking against external software, such as MPI and HDF5, and a long rat's tail of others (BLAS, FFTW, GSL, hwloc, LAPACK, OpenSSL, PAPI, PETSc, zlib). It is an unfortunate reality of current HPC programming that using libraries remains a very difficult task. Our current way of handling these in Cactus is two-fold: (1) there is a flexible mechanism to find, check for, or manually specify the location of external libraries (2) external libraries can be bundled in a Cactus thorn, and be built automatically (3) Cactus can choose automatically between these two I claim that the current mechanism does not work well, for at least the following reasons: - if Cactus builds an external library, it does so too often (once per configuration or executable, not once per system or compiler version) - the current convenience mechanism it overly complex, in the sense that people don't understand any more under what conditions an external library is built; Cactus does things differently from what people expect, and errors go unnoticed - bundling source code for external libraries makes for large thorns, even if it is not used - the current implementation is complex and difficult to maintain, for a variety of reasons that have been extensively discussed - there are several other (minor) problems that are nonetheless difficult to solve Furthermore, people have different preferences regarding building or using external libraries, ranging from making it simple for new users, to reducing download time/disk space, reducing build time, to obtaining the best possible performance. These conflicting goals (all valid!) make it difficult to work towards a better solution. To solve this problem, I make the following radical suggestion: >>> We stop building external libraries as part of building a Cactus executable. <<< It was a good and promising idea, but it failed to deliver; it is causing more problems than it solves. In the new system, there will be two kinds of external libraries: - like MPI or HDF5: an external library must be present that Cactus uses - like LORENE: we always build our own (more on this below) Clearly, we still must make it easy to install a missing external library. Building PETSc is not easy, and then pointing Cactus to a self-built PETSc is not easy either. There are various options to build an external library outside Cactus: (1) apt-get install or similar (for local machines) (2) module/softenv (on many HPC systems) (3) the ET maintainers build it, and make it publicly available, on a set of "important" machines (4) manual build (configure; make; make install) -- often not difficult, if brief (ten lines) of instructions are given (5) we provide thorns and parameter files that automate this, as in "exe/cactus_sim build-hdf5.par"; these would need to be run explicitly once per user on every system (6) using a professional package manager that doesn't require root access, such as Spack <https://github.com/LLNL/spack> Regarding thorns like LORENE: We should switch from distributing the LORENE source code as tarball to keeping the unpacked source code in the thorn, with a set of patches applied. In this way, LORENE can be built as regular Cactus thorn, together with other Cactus source code, and will be linked in a natural way -- no configuration magic necessary at all. All we need is a makefile that lists the LORENE source files. -erik -- Erik Schnetter <schnetter(a)gmail.com> http://www.perimeterinstitute.ca/personal/eschnetter/

3 3

Resolving two BH
by rahul kashyap 08 Jun '16

08 Jun '16

Hello Everyone, I'm working on binary black hole mergers with large mass ratio (20:1 or more). By using higher refinement level, I have been able to resolve the smaller BH, but it doesn't remain resolved even for a single time step. So far, I have tried changing the CarpetRegrid2::regrid_every and AHFinderDirect::find_every to as small as 1. Could someone please help me where I'm going wrong? I'm sure there are other problems too with my parameter file. I have attached the par file if anyone would be kind enough to give me further suggestions for this problem. Thanks, Rahul

5 7

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users