[Commits] [svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 189)

[roland.haas at physics.gatech.edu]

User: rhaas
Date: 2011/11/11 10:02 PM

Modified:
 /
  ET.tex

Log:
 work through Christian's suggestions until section '4. Core Technologies'
 
 some paper references would be easiest of someone would speak up if
 (a) they use many cores in there simulations (the more the better)
 (b) know the old papers introducing Cactus (one written by LSU, AEI & NCSA
     members)

File Changes:

Directory: /
============

File [modified]: ET.tex
Delta lines: +43 -32
===================================================================
--- ET.tex	2011-11-12 00:34:24 UTC (rev 188)
+++ ET.tex	2011-11-12 04:02:46 UTC (rev 189)
@@ -346,34 +346,42 @@
 
 \section{Core Technologies}
 
-The Einstein Toolkit consists of many components which, combined by the
-user, are utilized together to perform a full simulation. While all of these
-components have important tasks, a few stand out, and four
-are described in more detail below: the {\tt Cactus} framework, providing the 
-underlying infrastructure of which many other components make use,  
-adaptive mesh refinement drivers,
-without which most results could not be obtained in reasonable time,
-the Simulation Factory, because it simplifies the necessary supercomputer usage,
-and Kranc, which can generate code in a computer language from a high-level
-description in Mathematica.
+The Einstein Toolkit modules center around a set of core modules that provide
+basic functionality to create, deploy and manage a numerical simulation
+starting with code generation all to way to archiving of simulation
+results: (i) the {\tt Cactus} framework ``flesh'' provides the underlying
+infrastructure to build complex simulation codes out of independently
+developed modules and facillates communication between these modules. (ii) the
+adaptive mesh refinement driver, {\tt Carpet}, is build on top of {\tt Cactus}
+and provides problem independent adaptive mesh refinement support for
+simulations that need to resolve physics on length scales differing by many
+orders of magnitude, while relieving the scientist of the need to worry about
+internal details of the mesh refinement driver. (iii) {\tt Kranc}, which generates
+code in a computer language from a high-level description in Mathematica and
+(iv) the Simulation Factory, which provides a uniform, high-level interface to
+common operations, such submisson and restart of jobs, for a large number of
+compute clusters.
 
 \subsection{Cactus Framework}
 
 The {\tt Cactus}
   Framework~\cite{Cactuscode:web,Goodale:2002a,CactusUsersGuide:web} is
 an open source, modular, portable programming environment for
-collaborative HPC computing, primarily developed at Louisiana State University\@.
+collaborative HPC computing, primarily developed at Louisiana State University,
+and historically at the Albert Einstein Institute and the
+National Center for Supercomputing Applications~\todo{RH: for the senior
+people: is there a citable paper or somthing similar that docuemnts NCSA's
+involvement in Cactus?}.
 The {\tt Cactus} computational toolkit consists of general modules providing
 parallel drivers, coordinates, boundary conditions, interpolators,
 reduction operators, and efficient I/O in different data
 formats. Generic interfaces are used, making it possible to use
 external packages and improved modules which are immediately available
-to its users.  {\tt Cactus} is involved in the NSF Blue Waters consortium
-for petascale computing, and has funding from NSF SDCI to
-develop new application-level tools for performance and correctness.
+to its users.
 
 The structure of the {\tt Cactus} framework is completely modular, with
-only a very small core providing the interfaces between modules,
+only a very small core (the ``flesh'') providing the interfaces between
+modules,
 both at compile- and run-time. The {\tt Cactus} modules, called ``thorns'',
 may, and typically do, specify inter-module dependencies, e.g., to share or
 extend configuration information, common variables, or runtime parameters.
@@ -390,15 +398,16 @@
 The {\tt Cactus} Framework was developed by the
 numerical relativity community, and although it is a general component
 framework that supports different application domains its core user
-group has remained from numerical relativity. The {\tt Cactus} team has
-traditionally developed and supported a set of core modules for numerical
-relativity, as part of the \texttt{CactusEinstein} arrangement. Over the
-last few years however, the relevance of many of the modules has declined,
-and more and more of the basic infrastructure for numerical relativity
-has been provided by open modules distributed by research
-groups within the community. 
-The Einstein Toolkit now collects the widely used parts of CactusEinstein,
-combined with contributions from the community.
+group continues to be comprised of numerical relativists. 
+It is not surprising therefore, that one of the science modules provided in
+the Einstein Toolkit is a set of state of the art modules to simulate binary
+black hole mergers. All modules to simulate and analyse the data are provided
+out of the box. This set of modules also provides a way of testing the
+Einstein Toolkit modules in a production type simulation rather than synthetic
+test cases. Some of these modules have been developed specifically for the
+Einstein Toolkit while others are modules used in previous publications and
+have been contributed to the toolkit. In these cases the Einstein Toolkit
+provides documenation and best practise guidelines for the contributed modules.
 
 \subsection{Adaptive Mesh Refinement}
 
@@ -416,14 +425,14 @@
 The Einstein Toolkit offers two drivers, \codename{PUGH} and
 {\tt Carpet}. \codename{PUGH} provides domains consisting of a uniform 
 grid with Cartesian topology, and is highly scalable (up to more than
-130,000 cores on a Blue Gene/P~\cite{Cactuscode:BlueGene:web}.)
+130,000~\cite{Cactuscode:BlueGene:web}.)
 {\tt Carpet}~\cite{Schnetter:2003rb, Schnetter:2006pg,
   CarpetCode:web} provides multi-block methods and adaptive mesh
 refinement (AMR\@). Multi-block methods cover the domain with a set of
 (possibly distorted) blocks that exchange boundary information via techniques such as
 interpolation or penalty methods.\footnote{Although multi-block
   methods are supported by {\tt Carpet}, the Einstein Toolkit itself
-  does not yet
+  does not currently
   contain any multi-block coordinate systems.} The AMR capabilities
 employ the standard Berger-Oliger algorithm~\cite{Berger:1984zza} with
 subcycling in time.
@@ -473,7 +482,8 @@
 
 {\tt Carpet} is the main driver used today for {\tt Cactus}-based astrophysical
 simulations. {\tt Carpet} offers hybrid MPI/OpenMP parallelization and is
-used in production on up to several thousand cores.
+used in production on up to several thousand cores~\cite{Reisswig:2010cd,Lousto:2010ut}.
+\todo{if you use many cores in your runs: please speak up}
 \begin{figure}
   \centering
   \includegraphics[width=0.5\textwidth]{cactus-benchmarks/results-best}
@@ -491,15 +501,16 @@
 where \texttt{McLachlan} (see section \ref{sec:Kevol} below) solves
 the Einstein equations on a grid structure with
 nine levels of mesh refinement. This demonstrates excellent
-scalability up to more than ten thousand cores. (In production
-simulations, smaller and more complex grid structures and other
-necessary tasks reduce scalability by about a factor of ten.)
+scalability up to more than ten thousand cores. In production
+simulations, smaller and more complex grid structures, serial
+tasks related to online data analysis and other
+necessary tasks reduce scalability by up to a factor of ten.
 
 We estimate that,
 in 2010, about 7,000 core years of computing time (45 million core
 hours) were used via {\tt Carpet} by more than a dozen research groups
 world-wide. To date, more than 90 peer-reviewed publications and more
-than 15 student theses are based on {\tt Carpet}~\cite{CarpetCode:web}.
+than 15 student theses have been based on {\tt Carpet}~\cite{CarpetCode:web}.
 
 \subsection{Simulation Factory}
 
@@ -518,7 +529,7 @@
 
 The Simulation Factory~\cite{Thomas:2010aa, SimFactory:web} offers a
 set of abstractions for the tasks necessary to set up and successfully
-finish numerical simulations based on the {\tt Cactus} framework. These
+complete numerical simulations based on the {\tt Cactus} framework. These
 abstractions hide tedious low-level management operations, they
 capture ``best practices'' of experienced users, and they create a log
 trail ensuring repeatable and well-documented scientific results.