Commits

commits@lists.einsteintoolkit.org

2 participants
6997 discussions

[svn:einsteintoolkit] www/release-info/ (Rev. 491)
by ian.hinder＠aei.mpg.de 04 Apr '11

04 Apr '11

User: hinder Date: 2011/04/04 01:02 PM Added: /release-info/ datura__1_1.log, datura__2_1.log, datura__2_2.log Log: Add Datura results File Changes: Directory: /release-info/ ========================= File [added]: datura__1_1.log Delta lines: <Unavailable> File [added]: datura__2_1.log Delta lines: <Unavailable> File [added]: datura__2_2.log Delta lines: <Unavailable>

1 0

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 47)
by knarf＠cct.lsu.edu 04 Apr '11

04 Apr '11

User: knarf Date: 2011/04/04 09:54 AM Modified: / ET.tex Log: extend Base Modules File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +94 -22 =================================================================== --- ET.tex 2011-04-04 14:35:53 UTC (rev 46) +++ ET.tex 2011-04-04 14:54:56 UTC (rev 47) @@ -319,7 +319,7 @@ The Einstein Toolkit now collects the widely used parts of CactusEinstein, combined with contributions from the community. -\subsection{Adaptive Mesh Refinement\pages{1}} +\subsection{Adaptive Mesh Refinement\pages{1 Erik}} In Cactus, infrastructure capabilities such as memory management, parallelisation, time evolution, mesh refinement, and I/O are @@ -380,7 +380,7 @@ world-wide. To date, more than 90 peer-reviewed publication and more than 15 student theses are based on Carpet \cite{CarpetCode:web}. -\subsection{Simulation Factory\pages{1}} +\subsection{Simulation Factory\pages{1 Erik}} Today's supercomputers differ significantly in their hardware configuration, available software, directory structure, @@ -478,6 +478,10 @@ enforce on a technical level. Examples for these are the exact definitions of physical variables, their units, but also on the more technical level the variable names used for the physical quantities. +In addition, even distinct simulation codes typically use very similar +scheduling schemes. Conventions on the level of the scheduler can help +coordinating the order different modules are called. + The Einstein Toolkit provides modules with the sole purpose to declare commonly used variables, and to define their meaning and units. The latter is not strictly enforced, and instead documented as part of the @@ -485,8 +489,9 @@ following in more detail, because especially these quantities might be defined differently in various simulation codes. +\subsubsection{ADMBase} Relativistic spacetime evolution methods used within the Cactus framework use -different formulations, but essentially all are based on the ADM +different formulations, but essentially all are based on the $3+1$ ADM construction~\cite{Arnowitt:1962hi}. This makes this formulation the natural choice of a common denominator to exchange data between modules using different formulations. The following variables are @@ -499,30 +504,97 @@ between different modules, like which type of initial data should be setup or which evolution method should be used. +The variables provided by {\tt ADMBase} are: + +\begin{itemize} + \item + The 3-metric tensor, $g_{ij}$ + {\tt gxx}, {\tt gxy}, {\tt gxz},{\tt gyy}, {\tt gyz},{\tt gzz} + \item The extrinsic curvature tensor, $K_{ij}$ + {\tt kxx}, {\tt kxy}, {\tt kxz},{\tt kyy},{\tt kyz},{\tt kzz} + \item The lapse function, $\alpha$ + {\tt alp} + \item The shift vector $\beta^i$ + {\tt betax}, {\tt betay},{\tt betaz} +\end{itemize} + +The type of chosen initial data for a simulation is specified by the {\tt +initial\_data} (3-metric and extrinsic curvature), +{\tt initial\_lapse}, {\tt initial\_shift} parameters, and +parameters for their first time derivatives {\tt initial\_dtlapse} and +{\tt initial\_dtshift} respectively. +By default, {\tt ADMBase} initialises the 3-metric and extrinsic +curvature to Minkowski and the lapse to one. Initial data thorns +override these defaults by extending the parameters. + +Analogous to specifying initial data, evolution methods are chosen by +the {\tt evolution\_method} (3-metric and extrinsic curvature), +{\tt lapse\_evolution\_method}, {\tt shift\_evolution\_method}, +{\tt dtlapse\_evolution\_method} and {\tt dtshift\_evolution\_method} +parameters. {\tt ADMBase} does not evolve the +3-metric or extrinsic curvature, and holds the lapse and shift static. +Evolution thorns typically extend the ranges of these parameters and +contain the evolution code. + +\subsubsection{HydroBase} Similar to \codename{ADMBase}, the module \codename{HydroBase} defines the common basis for interactions between relativistic hydrodynamics modules. +HydroBase extends the CactusEinstein framework to include an interface for magnetohydrodynamics to +work within. HydroBase's main function is to store the primitive variables, common among +hydrodynamic simulations, commonly needed parameters, and schedule groups for the main +functions of a hydrodynamics code. +HydroBase only stores variables which +are common to most if not all hydrodynamics codes solving the Euler +equations, the so called primitive variables. These are also the variables which are needed +to couple to a spacetime solver and which are usually needed by analysis thorns. The usage of +a common set of variables by different hydrodynamics codes creates the possibility to share +parts of the code, e.g.\ initial data solvers or analysis routines. + It is using the conventions of the Valencia formulation~\cite{Marti:1991wi,Banyuls:1997zz,Ibanez:2001:godunov}. -In particular, it defines the rest-mass density $\varrho$, the gas -pressure $p$, internal energy density $\epsilon$, the -contravariant gas three velocity defined as -\begin{equation} - v^i = \frac{u^i}{\alpha u^0} + \frac{\beta^i}{\alpha} -\end{equation} -the electron -fraction $Y_e$, the temperature $T$, the specific entropy per -particle $s$ and the contravariant magnetic field vector $B^i$, defined -as -\begin{equation} - B^i = \frac{1}{\sqrt{4\pi}} n_{\nu} F^{*\nu i} -\end{equation} -in terms of the dual -\begin{equation} - F^{*\mu\nu} = \frac{1}{2}\varepsilon^{\mu\nu\alpha\beta}F_{\alpha\beta} -\end{equation} -to the Faraday tensor and the unit normal of the foliation of spacetime -$n^\mu$. +In particular, HydroBase defines the primitive variables (see~\cite{livrevgrfd} for +details): +\begin{itemize} + \item \verb|rho|: rest mass density $\varrho$ + \item \verb|press|: pressure $p$ + \item \verb|eps|: internal energy density $\epsilon$ + \item \verb|vel[3]|: contravariant gas/fluid three velocity $v^i$ defined as + \begin{equation} + v^i = \frac{u^i}{\alpha u^0} + \frac{\beta^i}{\alpha} + \end{equation} + in terms of the four-velocity $u^\mu$, lapse $\alpha$, and shift vector + $\beta^i$. + \item \verb|Y_e|: electron fraction $Y_e$ + \item \verb|temperature|: temperature $T$ + \item \verb|entropy|: specific entropy per particle $s$ + \item \verb|Bvec[3]|: contravariant magnetic field vector defined as + \begin{equation} + B^i = \frac{1}{\sqrt{4\pi}} n_{\nu} F^{*\nu i} + \end{equation} + in terms of the dual + $F^{*\mu\nu} = \frac{1}{2}\varepsilon^{\mu\nu\alpha\beta}F_{\alpha\beta}$ + to the Faraday tensor and the unit normal of the foliation of spacetime + $n^\mu$. +\end{itemize} +HydroBase also sets up scheduling blocks that organize the main functions which modules of a +hydrodynamics code may need. All of those scheduling blocks are optional, however if used, +they might simplify existing codes and make them more interoperable. HydroBase itself does +not schedule something inside most of the groups which it provides. + +Currently the scheduling blocks are: +\begin{itemize} + \item Initializing the primitive variables + \item Converting primitive variables to conservative variables + \item Calculating the right hand side (RHS) in the method of lines (MoL) + \item Setting and updating an excision mask + \item Applying boundary conditions +\end{itemize} + +In this way the initiation of the primitive variables, methods to recover the conservative +variables, and basic atmosphere handling can be implemented in different thorns while allowing +a central access point for analysis thorns. + \subsection{Initial Data\pages{4 Josh/Bruno}} The Einstein Toolkit contains many modules used to generate initial data for general relativistic simulations, including both vacuum and hydrodynamical

1 0

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 46)
by knarf＠cct.lsu.edu 04 Apr '11

04 Apr '11

User: knarf Date: 2011/04/04 09:35 AM Modified: / ET.tex Log: more on design and strategy File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +8 -5 =================================================================== --- ET.tex 2011-04-04 14:29:53 UTC (rev 45) +++ ET.tex 2011-04-04 14:35:53 UTC (rev 46) @@ -252,7 +252,7 @@ other areas of science and hereby providing an immediate application of their research. -\section{Design and Strategy\pages{1 Frank}} +\section{Design and Strategy\pages{0.5 Frank}} The mechanisms for the development and support of the Einstein Toolkit are designed to be open, transparent and community driven. All the source code, @@ -266,9 +266,11 @@ discussion about the toolkit takes place on an open mail list ({\tt users(a)einsteintoolkit.org}). The regular weekly meetings for the Einstein Toolkit are open and the community is invited to -participate, and minutes are recorded on the wiki\todo{check}. The Einstein Toolkit +participate, and minutes are recorded and publicly available. The Einstein Toolkit blog requires users to first request a login, but then can be freely posted to. Any user can post comments to entries already on the blog. +The community heavily uses an issue tracking system +({\tt trac.einsteintoolkit.org}), with submissions open to everyone. Despite the open design, some actions have to be restricted to a small group of maintainers. This is true for e.g. administrative tasks like the @@ -276,9 +278,10 @@ of spam. However, one of the most important tasks of an Einstein Toolkit Maintainer is also to review and apply patches sent by users, to ensure a high software quality level. Every substancial change or addition to -the toolkit has, by not enforced consensus, to be reviewed by another -Einstein Toolkit Maintainer, and is generally open for discussion on the -users mailing list. +the toolkit has to be reviewed by another Einstein Toolkit Maintainer, +and is generally open for discussion on the users mailing list. Despite this +not being enforced by technically it seems to work pretty well while at the +same time promoting development. \section{Core Technologies\pages{0.5 Frank}}

1 0

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 45)
by knarf＠cct.lsu.edu 04 Apr '11

04 Apr '11

User: knarf Date: 2011/04/04 09:29 AM Modified: / ET.tex Log: address todo File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +3 -3 =================================================================== --- ET.tex 2011-04-04 13:55:01 UTC (rev 44) +++ ET.tex 2011-04-04 14:29:53 UTC (rev 45) @@ -98,13 +98,13 @@ \section{Introduction\pages{1 Frank}} \comment{About 1 page} -\todo{ES: Mention that ET development receives NSF funding.} - Scientific progress in the field of numerical relativity has always been closely tied with the availability and ease-of-use of enabling software and computational infrastructure. This document describes the Einstein Toolkit, which aims at providing such an infrastructure, -developed openly and available freely. +developed openly and available freely, and is directly supported by +the National Science Foundation under the grant numbers +0903973/0903782/0904015 (CIGR). Now is a particularly exciting time for numerical relativity and relativistic astrophysics. Recent computational

1 0

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 44)
by schnetter＠cct.lsu.edu 04 Apr '11

04 Apr '11

User: eschnett Date: 2011/04/04 08:55 AM Modified: / ET.tex Log: Update Carpet and SimFactory sections File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +55 -23 =================================================================== --- ET.tex 2011-04-04 05:12:38 UTC (rev 43) +++ ET.tex 2011-04-04 13:55:01 UTC (rev 44) @@ -98,6 +98,8 @@ \section{Introduction\pages{1 Frank}} \comment{About 1 page} +\todo{ES: Mention that ET development receives NSF funding.} + Scientific progress in the field of numerical relativity has always been closely tied with the availability and ease-of-use of enabling software and computational infrastructure. This document describes @@ -314,13 +316,8 @@ The Einstein Toolkit now collects the widely used parts of CactusEinstein, combined with contributions from the community. -\subsection{Adaptive Mesh Refinement\pages{1 Erik}} +\subsection{Adaptive Mesh Refinement\pages{1}} -\todo{ES: add/check citations} -\todo{ES: remove names} -\todo{ES: describe grants in acknowledgements, also in introduction, - but not in individual sections} - In Cactus, infrastructure capabilities such as memory management, parallelisation, time evolution, mesh refinement, and I/O are delegated to a set of special \emph{driver} components. This helps @@ -333,33 +330,56 @@ program. The Einstein Toolkit offers two drivers, \emph{PUGH} and -\emph{Carpet}. PUGH provides domains consisting of a uniform Cartesian -grid, and is highly scalable (up to more than 130,000 cores on a Blue +\emph{Carpet}. PUGH provides domains consisting of a uniform +grid with Cartesian topology, and is highly scalable (up to more than +130,000 cores on a Blue Gene/P \todo{cite}). Carpet \cite{Schnetter:2003rb, Schnetter:2006pg, CarpetCode:web} provides multi-block methods and adaptive mesh refinement (AMR\@). Multi-block methods cover the domain with a set of -distorted Cartesian blocks that exchange information e.g.\ via +(possibly distorted) blocks that exchange boundary information e.g.\ via interpolation or penalty methods.\footnote{Although multi-block methods are supported by Carpet, the Einstein Toolkit does not yet contain any multi-block coordinate systems.} The AMR capabilities -employ the standard Berger-Oliger algorithm \todo{cite} with +employ the standard Berger-Oliger algorithm \cite{Berger84} with subcycling in time. +Adaptive mesh refinement means that the resolution in the simulation +domain is dynamically adapted to the current state of the simulation, +i.e.\ regions that require a higher resolution are covered with blocks +with a finer grid (typically by a factor of two); these are called a +\emph{refined level}. Finer grids can be also recursively refined +again. At regular intervals, the resolution requirements in the +simulation are re-evaluated, and the grid hierarchy is updated; this +step is called \emph{regridding}. + +Since a finer grid spacing also requires smaller time steps for +hyperbolic problems, the finer grids perform multiple time steps for +each coarse grid time step, leading to a recursive time evolution +pattern that is typical for Berger-Oliger AMR\@. If a simulation uses +e.g.\ 11 levels, then the resolutions (both in space and time) of the +the coarsest and finest levels differ by a factor of 1024. This +non-uniform time stepping leads to a certain complexity that is also +handled by the Carpet driver; for example, applying boundary +conditions to a fine level requires interpolation in space and time +from a coarser level. Outputting the solution at a time in between +coarse grid time steps also requires interpolation. These parallel +interpolation operations are implemented efficiently in Carpet, and +are applied automatically as specified in the execution schedule, +i.e.\ without requiring function calls in user code. + +\todo{ES: Add figure from Carpet paper explaining subcycling in time.} + Carpet is the main driver used today for Cactus-based astrophysical simulations. Carpet offers hybrid MPI/OpenMP parallelisation and is used in production on up to several thousand cores. We estimate that, in 2010, about 7,000 core years of computing time (45 million core hours) were used via Carpet by more than a dozen research groups world-wide. To date, more than 90 peer-reviewed publication and more -than 15 student theses are based on Carpet \todo{ES: update these numbers} -\cite{CarpetCode:web}. -% Carpet's continued development is overseen by -% E. Schnetter \todo{ES: do we want names here?} and is funded via several -% NSF awards \todo{ES: list them somewhere}. +than 15 student theses are based on Carpet \cite{CarpetCode:web}. -\subsection{Simulation Factory\pages{1 Erik/Michael}} +\subsection{Simulation Factory\pages{1}} -It is unfortunate that today's supercomputers differ significantly in +Today's supercomputers differ significantly in their hardware configuration, available software, directory structure, queueing system, queuing policiy, and many other user-visible properties. In addition, the system architectures and user interfaces @@ -382,13 +402,25 @@ types of potentially disastrous user errors are avoided, and different supercomputers can be used in a uniform manner. -Using the Simulation Factory, we are able to offer a tutorial for the -Einstein Toolkit \todo{cite} that lets new users download, configure, -build, and run full simulations of the coupled Einstein/relativistic -hydrodynamics equations on a supercomputer with a few simple commands. -The exact same set of commands can be used on all other supported -supercomputers to run the same simulation there. +For example, using the Simulation Factory, we are able to offer a +tutorial for the Einstein Toolkit \cite{EinsteinToolkit:web} that lets +new users download, configure, build, and run full simulations of the +coupled Einstein/relativistic hydrodynamics equations on a +supercomputer with a few simple commands. Users need no prior +knowledge about either the details of the architecture of a +supercomputer nor about the particular software configuration of this +supercomputer. The exact same set of SimFactory commands can be used +on all other supported supercomputers to run the same simulation +there. +The Simulation Factory supports and simplifies three kinds of +operations: +\begin{description} +\item[Remote Access] \todo{ES} +\item[Configuring and Building] \todo{ES} +\item[Submitting and Managing Simulations] \todo{ES} +\end{description} + \subsection{Kranc\pages{1 Ian}} \label{sec:kranc}

1 0

[svn:einsteintoolkit] www/release-info/ (Rev. 490)
by tanja.bode＠physics.gatech.edu 04 Apr '11

04 Apr '11

User: tbode Date: 2011/04/04 08:49 AM Modified: /release-info/ compile_tests Log: Add missing parentheses around gatech lines. File Changes: Directory: /release-info/ ========================= File [modified]: compile_tests Delta lines: +1 -1 =================================================================== --- release-info/compile_tests 2011-04-04 12:43:36 UTC (rev 489) +++ release-info/compile_tests 2011-04-04 13:49:13 UTC (rev 490) @@ -19,7 +19,7 @@ hostname | grep ^topf && cp ~/Cactus/simfactory/udb.pm simfactory/udb.pm hostname | grep '^phys44230\.physics\.gatech\.edu' && - cp ~/postdoc/udb.pm simfactory/udb.pm ; cp ~/postdoc/gatech-gcc.cfg simfactory/optionlists + ( cp ~/postdoc/udb.pm simfactory/udb.pm ; cp ~/postdoc/gatech-gcc.cfg simfactory/optionlists ) # main build ./simfactory/sim build sim --thornlist=../einsteintoolkit.th

1 0

[svn:einsteintoolkit] www/release-info/ (Rev. 489)
by schnetter＠cct.lsu.edu 04 Apr '11

04 Apr '11

User: eschnett Date: 2011/04/04 07:43 AM Added: /release-info/ hopper__1_1.log, hopper__2_1.log, hopper__2_2.log Log: Add results from Hopper at NERSC File Changes: Directory: /release-info/ ========================= File [added]: hopper__1_1.log Delta lines: <Unavailable> File [added]: hopper__2_1.log Delta lines: <Unavailable> File [added]: hopper__2_2.log Delta lines: <Unavailable>

1 0

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 43)
by knarf＠cct.lsu.edu 04 Apr '11

04 Apr '11

User: knarf Date: 2011/04/04 12:12 AM Modified: / ET.tex Log: some future work File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +24 -4 =================================================================== --- ET.tex 2011-04-02 02:03:03 UTC (rev 42) +++ ET.tex 2011-04-04 05:12:38 UTC (rev 43) @@ -1258,11 +1258,31 @@ \section{Future Work\pages{1 Frank}} -- MHD -- massively parallel AMR -- radiation transport -- advanced wave extraction +This paper illustrated the current state of the ``Einstein Toolkit'', +a collection of freely available and easy to use computational codes +for numerical relativity. However, there is room for improvement on both +the underlying infrastructure and the included physics. +One of the desirable additions of physics is a proper treatment of +radiation, in particular neutrinos, but possibly even some approximation +of emmission of electro-magnetic waves. Radiation transport is, even compared +to the full GRMHD problem, computationally very expensive, especially in +three dimensions. \todo{mention PetaCactus} + +\todo{mention advanced wave extraction} + +Another important goal is to increase the scalability of the Carpet AMR +infrastructure. It has been shown that good scaling is limited to less than +a few thousend proccesses, for some of the most used simulation scenarios. +In this case, work is already in progress to eliminate this bottle-neck. +On the other hand, a production simulation is typically composed from a large +number of components, and all of them have to scale well to achieve overall +good performance. This is not an easy problem, as most of the module authors +are neither computer scientists nor had they extensive training in parallel +development and profiling techniques. Close collaboration with such experts +has been shown to be very fruitful in the past and is definitely also planned +for the forseeable future. + \section*{Acknowledgments\pages{0.5 All}} \todo{Frank, ADDME} The Einstein Toolkit is directly supported by the National Science Foundation

1 0

[svn:einsteintoolkit] www/release-info/ (Rev. 488)
by schnetter＠cct.lsu.edu 03 Apr '11

03 Apr '11

User: eschnett Date: 2011/04/03 10:22 PM Added: /release-info/ lonestar__1_1.log, lonestar__2_1.log, lonestar__2_2.log Log: Add results from Lonestar File Changes: Directory: /release-info/ ========================= File [added]: lonestar__1_1.log Delta lines: <Unavailable> File [added]: lonestar__2_1.log Delta lines: <Unavailable> File [added]: lonestar__2_2.log Delta lines: <Unavailable>

1 0

[svn:einsteintoolkit] www/release-info/ (Rev. 487)
by diener＠cct.lsu.edu 03 Apr '11

03 Apr '11

User: diener Date: 2011/04/03 08:50 PM Added: /release-info/ amarok__1_1.log, amarok__1_2.log, amarok__2_1.log, amarok__2_2.log Log: Results from Amarok File Changes: Directory: /release-info/ ========================= File [added]: amarok__1_1.log Delta lines: <Unavailable> File [added]: amarok__1_2.log Delta lines: <Unavailable> File [added]: amarok__2_1.log Delta lines: <Unavailable> File [added]: amarok__2_2.log Delta lines: <Unavailable>

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Commits