[Commits] [svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 44)

[schnetter at cct.lsu.edu]

User: eschnett
Date: 2011/04/04 08:55 AM

Modified:
 /
  ET.tex

Log:
 Update Carpet and SimFactory sections

File Changes:

Directory: /
============

File [modified]: ET.tex
Delta lines: +55 -23
===================================================================
--- ET.tex	2011-04-04 05:12:38 UTC (rev 43)
+++ ET.tex	2011-04-04 13:55:01 UTC (rev 44)
@@ -98,6 +98,8 @@
 \section{Introduction\pages{1 Frank}}
 \comment{About 1 page}
 
+\todo{ES: Mention that ET development receives NSF funding.}
+
 Scientific progress in the field of numerical relativity has always
 been closely tied with the availability and ease-of-use of enabling
 software and computational infrastructure. This document describes
@@ -314,13 +316,8 @@
 The Einstein Toolkit now collects the widely used parts of CactusEinstein,
 combined with contributions from the community.
 
-\subsection{Adaptive Mesh Refinement\pages{1 Erik}}
+\subsection{Adaptive Mesh Refinement\pages{1}}
 
-\todo{ES: add/check citations}
-\todo{ES: remove names}
-\todo{ES: describe grants in acknowledgements, also in introduction,
-  but not in individual sections}
-
 In Cactus, infrastructure capabilities such as memory management,
 parallelisation, time evolution, mesh refinement, and I/O are
 delegated to a set of special \emph{driver} components. This helps
@@ -333,33 +330,56 @@
 program.
 
 The Einstein Toolkit offers two drivers, \emph{PUGH} and
-\emph{Carpet}. PUGH provides domains consisting of a uniform Cartesian
-grid, and is highly scalable (up to more than 130,000 cores on a Blue
+\emph{Carpet}. PUGH provides domains consisting of a uniform 
+grid with Cartesian topology, and is highly scalable (up to more than
+130,000 cores on a Blue
 Gene/P \todo{cite}). Carpet \cite{Schnetter:2003rb, Schnetter:2006pg,
   CarpetCode:web} provides multi-block methods and adaptive mesh
 refinement (AMR\@). Multi-block methods cover the domain with a set of
-distorted Cartesian blocks that exchange information e.g.\ via
+(possibly distorted) blocks that exchange boundary information e.g.\ via
 interpolation or penalty methods.\footnote{Although multi-block
   methods are supported by Carpet, the Einstein Toolkit does not yet
   contain any multi-block coordinate systems.} The AMR capabilities
-employ the standard Berger-Oliger algorithm \todo{cite} with
+employ the standard Berger-Oliger algorithm \cite{Berger84} with
 subcycling in time.
 
+Adaptive mesh refinement means that the resolution in the simulation
+domain is dynamically adapted to the current state of the simulation,
+i.e.\ regions that require a higher resolution are covered with blocks
+with a finer grid (typically by a factor of two); these are called a
+\emph{refined level}. Finer grids can be also recursively refined
+again. At regular intervals, the resolution requirements in the
+simulation are re-evaluated, and the grid hierarchy is updated; this
+step is called \emph{regridding}.
+
+Since a finer grid spacing also requires smaller time steps for
+hyperbolic problems, the finer grids perform multiple time steps for
+each coarse grid time step, leading to a recursive time evolution
+pattern that is typical for Berger-Oliger AMR\@. If a simulation uses
+e.g.\ 11 levels, then the resolutions (both in space and time) of the
+the coarsest and finest levels differ by a factor of 1024. This
+non-uniform time stepping leads to a certain complexity that is also
+handled by the Carpet driver; for example, applying boundary
+conditions to a fine level requires interpolation in space and time
+from a coarser level. Outputting the solution at a time in between
+coarse grid time steps also requires interpolation. These parallel
+interpolation operations are implemented efficiently in Carpet, and
+are applied automatically as specified in the execution schedule,
+i.e.\ without requiring function calls in user code.
+
+\todo{ES: Add figure from Carpet paper explaining subcycling in time.}
+
 Carpet is the main driver used today for Cactus-based astrophysical
 simulations. Carpet offers hybrid MPI/OpenMP parallelisation and is
 used in production on up to several thousand cores. We estimate that,
 in 2010, about 7,000 core years of computing time (45 million core
 hours) were used via Carpet by more than a dozen research groups
 world-wide. To date, more than 90 peer-reviewed publication and more
-than 15 student theses are based on Carpet \todo{ES: update these numbers}
-\cite{CarpetCode:web}.
-% Carpet's continued development is overseen by
-% E. Schnetter \todo{ES: do we want names here?} and is funded via several
-% NSF awards \todo{ES: list them somewhere}.
+than 15 student theses are based on Carpet \cite{CarpetCode:web}.
 
-\subsection{Simulation Factory\pages{1 Erik/Michael}}
+\subsection{Simulation Factory\pages{1}}
 
-It is unfortunate that today's supercomputers differ significantly in
+Today's supercomputers differ significantly in
 their hardware configuration, available software, directory structure,
 queueing system, queuing policiy, and many other user-visible
 properties. In addition, the system architectures and user interfaces
@@ -382,13 +402,25 @@
 types of potentially disastrous user errors are avoided, and different
 supercomputers can be used in a uniform manner.
 
-Using the Simulation Factory, we are able to offer a tutorial for the
-Einstein Toolkit \todo{cite} that lets new users download, configure,
-build, and run full simulations of the coupled Einstein/relativistic
-hydrodynamics equations on a supercomputer with a few simple commands.
-The exact same set of commands can be used on all other supported
-supercomputers to run the same simulation there.
+For example, using the Simulation Factory, we are able to offer a
+tutorial for the Einstein Toolkit \cite{EinsteinToolkit:web} that lets
+new users download, configure, build, and run full simulations of the
+coupled Einstein/relativistic hydrodynamics equations on a
+supercomputer with a few simple commands. Users need no prior
+knowledge about either the details of the architecture of a
+supercomputer nor about the particular software configuration of this
+supercomputer. The exact same set of SimFactory commands can be used
+on all other supported supercomputers to run the same simulation
+there.
 
+The Simulation Factory supports and simplifies three kinds of
+operations:
+\begin{description}
+\item[Remote Access] \todo{ES}
+\item[Configuring and Building] \todo{ES}
+\item[Submitting and Managing Simulations] \todo{ES}
+\end{description}
+
 \subsection{Kranc\pages{1 Ian}}
 \label{sec:kranc}