Commits

commits@lists.einsteintoolkit.org

2 participants
6997 discussions

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 179)
by jfaber＠einsteintoolkit.org 08 Nov '11

08 Nov '11

User: jfaber Date: 2011/11/08 10:29 PM Modified: / ET.tex Log: A few more edits to the Examples section. I'm pretty much done for now, the paper looks very good to me. File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +31 -32 =================================================================== --- ET.tex 2011-11-08 17:31:14 UTC (rev 178) +++ ET.tex 2011-11-09 04:29:46 UTC (rev 179) @@ -2645,10 +2645,10 @@ The examples in the previous subsections did not include the evolution of matter within a relativistic spacetime. One interesting test of a coupled matter-spacetime evolution is to measure the eigenfrequencies of a stable TOV -star~\todo{cite papers also doing this}. These eigenfrequencies can be compared +star~(see, e.g.,~\cite{Gourgoulhon:1991aa,Romero:1996aa,Shibata:1998sg,Font:2001ew,Shibata:2003iy}). \todo{Do we need more refs? Five is decent for an e.g.} These eigenfrequencies can be compared to values known from linear perturbation theory. -This test uses an equilibrium configuration of a self-gravitating fluid +We begin our simulations with a self-gravitating fluid sphere, described by a polytropic equation of state. This one-dimensional solution is obtained by the code described in section~\ref{sec:TOVSolver}, and is interpolated on the three-dimensional, computational evolution grid. @@ -2656,14 +2656,14 @@ \codename{McLachlan} and the hydrodynamics evolution system implemented in \codename{GRHydro}. -Specifically for this test, we set up a stable TOV star described by a +For the test described here, we set up a stable TOV star described by a polytropic equation of state $p=K\rho^\Gamma$ with $K=100$ and $\Gamma=2$, and an initial central density of $\rho_c=1.28\times10^{-3}$. This model can be taken to represent a non-rotating NS with a mass of $M=1.4\mathrm{M}_\odot$. The computational domain is a cube of length $640\mathrm{M}$ with a base resolution of $2\mathrm{M}$ ($4\mathrm{M}$, $8\mathrm{M}$) in each dimension. Four additional grids refine the region -around the star located at the origin, each doubling the resolution, with sizes +around the star centered at the origin, each doubling the resolution, with sizes of $120\mathrm{M}$, $60\mathrm{M}$, $30\mathrm{M}$ and $15\mathrm{M}$, resulting in a resolution of $0.125\mathrm{M}$ ($0.25\mathrm{M}$, $0.5\mathrm{M}$) across the entire star. @@ -2691,13 +2691,13 @@ hydrodynamics simulation, compared to the corresponding frequencies as obtained with perturbative techniques, kindly provided to use by Kentaro Takami and computed using the method described in~\cite{Yoshida:1999vj}. The PSD was computed -using the entire time series of the high-resolution run, removing the linear -trend, averaging over Hanning windows overlapping half the signal length after -padding the signal to five time it's length. The agreement of the frequencies -of the fundamental mode and the first three overtones is clearly visible, but -we are not able to show the same for higher overtones, at this resolution. -We expect to be able to resolve higher overtones when using even higher -resolution, but did not pursue further because of the involved computational +using the entire time series of the high-resolution run, by removing the linear +trend and averaging over Hanning windows overlapping half the signal length after +padding the signal to five time its length. The agreement of the + fundamental mode and first three overtone frequencies is clearly visible, but +are limited beyond this by the finite numerical resolution. +Higher overtones should be measurable with higher +resolution, but at substantial computational cost. \begin{figure} @@ -2718,7 +2718,7 @@ code lies between $1$ and $2$. It cannot be higher than $2$ due to the directional flux-split algorithm which is of second order. Depending on solution itself, the hydrodynamics code is only of first order in particular -regions, e.g., at extrema (like the center of the star), or at the star +regions, e.g., at extrema (like the center of the star), or at the stellar surface. Figure~\ref{fig:tov_ham_conv} shows the order of convergence of the Hamiltonian @@ -2736,7 +2736,7 @@ $r=5\mathrm{M}$. The observed convergence order of about $1.5$ at the center of the star is lower then the general second order of the hydrodynamics evolution scheme. This is expected because the scheme's convergence rate drops to first - order at extrema or shocks, like the stellar center or the star surface. + order at extrema or shocks, like the stellar center or surface. Consequently, the observed convergence order about half-way between the stellar center and surface is higher than $1.5$, but most of the time below $2$.} @@ -2744,13 +2744,13 @@ \subsection{Collapse\pages{2 Christian, Roland}} \label{sec:collapse_example} -The previous examples dealt either with a preexisting black hole (BBH) or -with a smooth singularity free spacetime (TOV oscillations). The evolution +The previous examples dealt either with preexisting BHs, either single or in a binary, or +with a smooth singularity free spacetime, as in the case of the TOV star. The evolution codes in the toolkit however are also able to handle the dynamic formation of a singularity as a star collapses -into a black hole. As a simple example of this process we study the collapse -of a non-rotating TOV star into a black hole. We create initial data as in -section~\ref{sec:tov_oscilations} using $\rho_c=3.154e-3$ and $K_{ID} = 100$, +into a BH, and as a simple example of this process we study the collapse +of a non-rotating TOV star. We create initial data as in +section~\ref{sec:tov_oscillations} using $\rho_c=3.154e-3$ and $K_{ID} = 100$, $\Gamma = 2$ yielding a star of mass $1.67\,M_\odot$. As is common in these situations~\cite{Baiotti:2005vi} we trigger collapse by reducing the pressure support after initial data has been constructed @@ -2779,7 +2779,7 @@ The outermost box has a radius of $R_0 = 204.8\,M_\odot$ and a resolution of $3.2\,M_\odot$ ($2.4\,M_\odot$, $1.6\,M_\odot$, $0.6\,M_\odot$ for higher convergence levels). -Around the star which is centered on the origin we stack $5$ extra boxes of +Around the star, centered on the origin, we stack $5$ extra boxes of approximate size $8\times2^\ell\,M_\odot$ for $0 \le \ell \le 4$, where the resolution on each finer % RH: there is a likely typo in the paramter file which creates boxes of @@ -2819,7 +2819,7 @@ approximate location of the star surface and the circumferential radius of the apparent horizon once a horizon is found in the simulation. Clearly the apparent -horizon is found at approximately the same time as the star's size approaches +horizon is first found at approximately the same time as when the star's size approaches its Schwarzschild radius. In Figure~\ref{fig:tov_collapse_H_convergence_at0} we display the convergence factor for the Hamiltonian constraint at the center of the collapsing star. @@ -2880,26 +2880,26 @@ \section{Conclusion and Future Work} In this article, we described the Einstein Toolkit, a collection -of freely available and easy to use computational codes for numerical +of freely available and easy-to-use computational codes for numerical relativity and relativistic astrophysics. The code details and example results present in this article represent the state of the Einstein Toolkit in its release ET\_2011\_05 ``Curie,'' released on April 21, 2011. -What was presented here is but a snapshot of the Einstein Toolkit's -ongoing development whose ultimate goal it is to provide an +The work presented here is but a snapshot of the Einstein Toolkit's +ongoing development, whose ultimate goal it is to provide an open-source set of robust baseline codes to realistically and reproducibly model the whole spectrum of relativistic astrophysical phenomena including, but not limited to, isolated black holes and neutron stars, binary black hole coalescence in vacuum and gaseous -environs, double neutron star and neutron star -- black hole mergers, +environments, double neutron star and neutron star -- black hole mergers, core-collapse supernovae, and gamma-ray bursts. For this, much future work towards including proper treatments of magnetic fields, more complex equations of state, nuclear reactions, neutrinos, and photons will be necessary and will need to be matched by improvements in infrastructure (e.g., more flexible AMR on general -grids) and computing hardware for the required fully coupled 3D, +grids) and computing hardware for the required fully coupled 3-D, multi-scale, multi-physics simulations to become reality. %%% One of the desirable additions of physics is a proper treatment of @@ -2922,18 +2922,17 @@ whole community by integrating their CCE routines into the Einstein Tookit release 2011\_11 ``Maxwell,'' which will be described elsewhere. -A second example for a much needed improvement of an existing method -is to transition to cell-centered AMR for GR hydrodynamic simulations, +A second much needed improvement of our existing methods +is a transition to cell-centered AMR for GR hydrodynamic simulations, which would allow for exact flux conservation across AMR interfaces via a refluxing step that adjusts coarse and/or fine grid fluxes for consistency (e.g., \cite{Berger:1984zza}). This is also a prerequisite for the constrained transport method \cite{Toth:00} for ensuring the divergence-free condition for the magnetic field in a future -implementation of GRMHD within the Einstein Toolkit. Work towards +implementation of GRMHD within the Einstein Toolkit. Work towards cell-centered AMR, refluxing, and GRMHD is underway and will be reported in a future publication. - %%% CDO: We have not even talked about the current MHD implementation, %%% how can we talk about its improvement? %%% @@ -2949,7 +2948,7 @@ Yet another important goal is to increase the scalability of the {\tt Carpet} AMR infrastructure. As we have shown, good scaling is -limited to only a few thousand processes for some of the most used +limited to only a few thousand processes for some of the most widely used simulation scenarios. Work is in progress to eliminate this bottle-neck. On the other hand, a production simulation is typically composed of a large number of components, and even analysis and I/O @@ -2968,8 +2967,8 @@ National Science Foundation under the grant numbers 0903973/0903782/0904015 (CIGR\@). Related grants contribute directly and indirectly to the success of CIGR, including NSF OCI-0721915, NSF -OCI-0725070, NSF OCI-0905046, and NSF OCI 0941653, NSF -AST-0855535, and NASA 08-ATFP08-0093. Results presented in this article were obtained through +OCI-0725070, NSF OCI-0832606, NSF OCI-0905046, and NSF OCI-0941653, NSF +AST-0855535, NSF DMS-0820923, and NASA 08-ATFP08-0093. Results presented in this article were obtained through computations on the Louisiana Optical Network Initiative under allocation loni\_cactus05 and loni\_numrel06, as well as on the NSF Teragrid under allocations TG-MCA02N014 and TG-PHY060027N, DOE repository m152, HLRB at

1 0

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 178)
by jfaber＠einsteintoolkit.org 08 Nov '11

08 Nov '11

User: jfaber Date: 2011/11/08 11:31 AM Modified: / ET.tex Log: BH examples sections edited, and some of RIT's acknowledgments added File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +69 -64 =================================================================== --- ET.tex 2011-11-08 04:25:03 UTC (rev 177) +++ ET.tex 2011-11-08 17:31:14 UTC (rev 178) @@ -2276,14 +2276,15 @@ \label{fig:rot180-grid} \end{figure} -\section{Examples\pages{0.5 Frank}} -\todo{Will be written once we know which examples we have} +\section{Examples} +\todo{Update if necessary} To demonstrate the properties of the code and its capabilities, we have used it to simulate common astrophysical configurations of interest. Given the community-oriented direction of the project, the parameter files required to launch these simulations and a host of others are included and documented in the code releases, along with the datafiles produced by a representative set of simulation parameters to allow for code validation and confirmation of correct code performance on new platforms and architetures. As part of the internal validation process, nightly builds are checked against a set of benchmarks to ensure that consistent results are generated with the inclusion of all new commits to the code. The performance of the Toolkit for vacuum configurations is demonstrated through evolutions of single, rotating BHs and the merger of binary black hole configurations (sections~\ref{sec:1bh-example} and \ref{sec:bbh-example}, respectively). Linear oscillations about equilibirum for an isolated NS are discussed in section~\ref{sec:tov_oscillations}, and the collapse of a NS to a BH, including dynamical formation of a horizon, in section~\ref{sec:collapse_example}. Finally, to show a less traditional application of the code, we show its ability to perform cosmological simulations by evolving a Kasner spacetime (see section~\ref{sec:cosmology}). -\subsection{Spinning BH\pages{2 Peter}} + +\subsection{Spinning BH} \label{sec:1bh-example} As a first example we perform simulations of a single distorted rotating BH. We use \codename{TwoPunctures} to set up initial data for a single @@ -2339,8 +2340,8 @@ \begin{figure} \includegraphics[width=0.33\textwidth,angle=-90]{examples/kerr/figs/waves_l4} \includegraphics[width=0.33\textwidth,angle=-90]{examples/kerr/figs/waves_l4_conv} - \caption{The right plot shows the real part of the extracted - $\ell =4, m=0$ mode of $\Psi_4$ as function of time from the high + \caption{Real part of the extracted + $\ell =4, m=0$ mode of $\Psi_4$ as function of time (left plot) for the high (solid red curve), medium (long dashed green curve) and low (short dashed blue curve) resolution runs. The extraction was done at $R=30M$. The left @@ -2363,21 +2364,21 @@ to perfect fourth order convergence. In addition to the modes shown in Figure~\ref{fig:kerr_waves} and -\ref{fig:kerr_waves_l4} we can mention that the extracted $\ell =4, m=4$ mode +\ref{fig:kerr_waves_l4} we note that the extracted $\ell =4, m=4$ mode is non-zero due to truncation error, but shows fourth order convergence to zero with resolution (this mode is not present in the initial data and is not excited during the evolution). Other modes are zero to round-off due to symmetries at all resolutions. -Since there is non-trivial gravitational wave content in the initial data -the mass of the BH changes when evolved. In Figure~\ref{fig:ah_mass} +Since there is non-trivial gravitational wave content in the initial data, +the mass of the BH changes during its evolution. In figure~\ref{fig:ah_mass} we show in the left plot the irreducible mass as calculated by \codename{AHFinderDirect} as function of time at the low (red solid curve), medium (green long dashed curve) and high (blue short dashed curve) resolutions. \begin{figure} \includegraphics[width=0.33\textwidth,angle=-90]{examples/kerr/figs/ah_mass} \includegraphics[width=0.33\textwidth,angle=-90]{examples/kerr/figs/ah_mass_conv} - \caption{The right plot shows the irreducible mass of the apparent horizon + \caption{The left plot shows the irreducible mass of the apparent horizon as function of time at low (red solid curve), medium (green long dashed curve) and high (blue short dashed curve) resolutions. The inset is a zoom in on the $y$-axis to more clearly show the differences between the resolutions. The @@ -2397,10 +2398,10 @@ (green long dashed curved) as well as the scaled difference between the high and medium resolutions for fourth order (blue short dashed curve) and third order (pink dotted curve). The convergence is almost perfectly -fourth order until $T=50M$ then better than fourth order until $T=60M$ and +fourth order until $T=50M$, then better than fourth order until $T=60M$, and finally between third order and fourth order for the remainder of the -evolution. The lack of perfect fourth order convergence at late time must -be due to non-convergent errors from the puncture being able to propagate +evolution. The lack of perfect fourth-order convergence at late times may be attributed + to non-convergent errors from the puncture propagating to the horizon location at the lowest resolution. Finally, in Figure~\ref{fig:ah_mass_spin} we show the total @@ -2424,13 +2425,12 @@ right plot of Figure~\ref{fig:ah_mass_spin} by the fact that the pink (dotted) curve (the high resolution result scaled by a factor of $1.78$ for second order convergence to the resolution of the medium resolution) and the -green (long dashed) curve are on top of each other. The reason for the -second order convergence in the spin lies in the fact that thorn -\codename{QuasiLocalMeasures} uses an algorithm which is -only second order accurate overall. The increase of about 0.22\% in the mass of the +green (long dashed) curve are on top of each other. Since the +\codename{QuasiLocalMeasures} thorn uses an algorithm which is +only second-order accurate overall, this is the expected result. The increase of about 0.22\% in the mass of the BH is caused solely by the increase in the irreducible mass. -\subsection{BHB\pages{2 Bruno}} +\subsection{BH Binary} \label{sec:bbh-example} %BCM: DONE:ID parameters, TP number of collocation points, grid structure, %coordinate domain, symmetries explored, description of the tracks, @@ -2439,21 +2439,23 @@ %analysis were performed, table with remnant AH parameters: final %spin, energy-momentum radiated with error bars estimated by the %difference between the values of the highest resolutions -\BCM{TODO: list the possible reasons we are not observing -the convergence rate expected.} +%JF: We have convergence plots before, so I'm not sure this is required for this section too +%\BCM{TODO: list the possible reasons we are not observing +%the convergence rate expected.} + To demonstrate the performance in the code for a problem of wide current scientific interest, we have evolved a non-spinning equal-mass BH binary system. -The initial data was prepared so that the binary would be +The initial data represent a binary in a quasi-circular orbit, with an initial separation chosen -to be $r=6M$ so we could track the later inspiral, +to be $r=6M$ so we may track the later inspiral, plunge, merger and ring down phases of the binary evolution. Table~\ref{table:BHB_ID} provides more details about the initial binary parameters used to generate the initial data. The \codename{TwoPunctures} module uses these initial parameters to solve \ref{eq:twopunc_u}, the elliptic Hamiltonian constraint for -the regular component of the conformal factor (see Sec.~\ref{sec:twopunctures}). +the regular component of the conformal factor (see section~\ref{sec:twopunctures}). The spectral solution for this example was determined by using $[n_A,n_B,n_{\phi}]=[28,28,14]$ collocation points, and, along with the Bowen-York analytic solution for the @@ -2491,16 +2493,16 @@ decomposition takes place. Figure~\ref{fig:tracks_waveform} shows the two puncture tracks -throughout all phases of the binary evolution. Tracks are +throughout all phases of the binary evolution, provided by the \codename{PunctureTracker} module. In the same plot we have recorded the intersection of the apparent horizon $2$-surface with the $z=0$ plane every time interval $t=10M$ during the evolution. A common horizon is first observed at $t=116M$. These apparent horizons were found by the \codename{AHFinderDirect} module and their -radius and location information stored into a $2$-surface of +radius and location information stored as a $2$-surface with spherical topology by the \codename{SphericalSurface} module. -The irreducible mass and dimensionless spin of the remnant were +The irreducible mass and dimensionless spin of the merged BH were calculated by the \codename{QuasiLocalMeasures} module, and were found to be $0.647 M$ and $-0.243 M^{-2}$, respectively. @@ -2509,9 +2511,9 @@ $\Psi_4$ in term of the metric components and its derivatives; these were computed to be $4$-th order accurate in this example. The second module, \codename{Multipole}, interpolates the -Weyl scalars into spheres with centers and radii specified by +Weyl scalars onto spheres with centers and radii specified by the user, and performs a spherical harmonic multipole -mode decomposition on these spherical surfaces. +mode decomposition. Figure~\ref{fig:tracks_waveform} shows the real and imaginary parts of the ($l=2$, $m=2$) mode for $\Psi_4$ extracted on a sphere centered at the origin at $R_{\rm obs} = 60M$. @@ -2530,55 +2532,58 @@ In order to evaluate the convergence of the numerical solution, we ran five simulations with different resolutions, and focus our analysis on the convergence -of the Weyl scalar $\Psi_4$ phase and amplitude. +of the phase and amplitude of the Weyl scalar $\Psi_4$. The mesh spacings adopted for the coarser grid in the AMR hierarchy for these different runs were $\{h_{\rm low},h_{\rm med},h_{\rm medh},h_{\rm high},h_{\rm higher}\} =\{2.0M,1.5M,1.25M,1.0M,0.75M\}$, respectively, while the finer grid spacings can be easily found by dividing -them by $2^{\rm level}$. In the case of this example, this -results in +them by $2^k$ for the $k$th level of mesh refinement.For this example, we set $\{h^f_{\rm low},h^f_{\rm med},h^f_{\rm medh},h^f_{\rm high},h^f_{\rm higher}\} =\{3.125M,2.344M,1.953M,1.563M,1.172M\}\times 10^{-2}$ for the finest grid in the different AMR hierarchies, respectively. -\BCM{Convergence factor text should be moved elsewhere -- starts here.} -We may define the convergence factor as the order of the Richardson extrapolation, +%JF - we assume this in the rpevious section with an equation, and can probably skip it here. +%\BCM{Convergence factor text should be moved elsewhere -- starts here.} +%We may define the convergence factor as the order of the Richardson extrapolation, % -\begin{equation} -Q(t) = \frac{||u^{h_{\rm low}}-u^{h_{\rm med}}||} - {||u^{h_{\rm med}}-u^{h_{\rm high}}||}, -\end{equation} -where $||\cdot||$ refers to an appropriate norm, and -$h_{\rm low}$, $h_{\rm med}$ and $h_{\rm high}$ correspond to grid -spacings of low, medium and high resolutions, respectively. -\BCM{stops here} +%\begin{equation} +%Q(t) = \frac{||u^{h_{\rm low}}-u^{h_{\rm med}}||} + % {||u^{h_{\rm med}}-u^{h_{\rm high}}||}, +%\end{equation} +%where $||\cdot||$ refers to an appropriate norm, and +%$h_{\rm low}$, $h_{\rm med}$ and $h_{\rm high}$ correspond to grid +%spacings of low, medium and high resolutions, respectively. +%\BCM{stops here} + Here, we consider the phase $\phi(t)$ and the amplitude $A(t)$ of the Weyl scalar $\Psi_4$ at $R_{\rm obs}=60M$. In order to take differences between the numerical values at two different grid resolutions, we use -an $8$-th order accurate Lagrange operator~\footnote{ +an $8$-th order accurate Lagrange operator to interpolate the higher-accuracy finite difference solution +into the immediately coarser grid. We have experimented with $4$-th and $6$-th order as well, to evaluate the level of noise these interpolations could -potentially introduce. We did not observe any noticeable -difference and we decided to use the higher order option.} -to interpolate the higher-accuracy finite difference solution -into the immediately coarser grid. -\BCM{Move this elsewhere too -- starts here} -Since the grid spacings -does not follow a $2:1$ ratio, the convergence factor has -a different functional form than the usual $2^p$ for a $p$-th accurate -finite difference scheme: +potentially introduce, but did not observe any noticeable +difference and we report here on results from the higher-order option. + +%JF: Do we need this at all, if we don't show a traditional convergence plot? +%\BCM{Move this elsewhere too -- starts here} +%Since the grid spacings +%does not follow a $2:1$ ratio, the convergence factor has +%a different functional form than the usual $2^p$ for a $p$-th accurate +%finite difference scheme: % -\begin{equation} -Q = \frac{b^p - a^p}{a^p - 1}, -\end{equation} -where $h_{\rm high}=h$, $h_{\rm med}=a\times h$ and -$h_{\rm low} = b \times h$. In our case, $a$ and $b$ -are $1.25$ and $1.5$, respectively. A $4$-th order convergent -scheme for our grid spacings would give then a convergence -factor of $Q = 1.818$, while a $6$-th and $8$-th order one, -$2.691$ and $3.965$, respectively. -\BCM{stops here. Remember to amend the text appropriately} +%\begin{equation} +%Q = \frac{b^p - a^p}{a^p - 1}, +%\end{equation} +%where $h_{\rm high}=h$, $h_{\rm med}=a\times h$ and +%$h_{\rm low} = b \times h$. In our case, $a$ and $b$ +%are $1.25$ and $1.5$, respectively. A $4$-th order convergent +%scheme for our grid spacings would give then a convergence +%factor of $Q = 1.818$, while a $6$-th and $8$-th order one, +%$2.691$ and $3.965$, respectively. +%\BCM{stops here. Remember to amend the text appropriately} + In Figure~\ref{fig:amp_phs_convergence}, we show the convergence of the amplitude and phase of the Weyl scalar by plotting the logarithm of the absolute value of the differences between two levels @@ -2593,7 +2598,7 @@ wave frequency reaches $\omega=0.2/M$. We follow a community standard, agreed to over the course of the NRAR\cite{NRAR:web} collaboration, that constrains the numerical resolution so that the accumulated phase error is not -larger than $0.05$ radians at a gravitational wave frequency around +larger than $0.05$ radians at a gravitational wave frequency of $\omega=0.2/M$. From the plot, we can assert that the phase error between the higher and high resolutions, and the one between high and medium-high resolutions satisfy this criterion, while the phase error between @@ -2963,11 +2968,11 @@ National Science Foundation under the grant numbers 0903973/0903782/0904015 (CIGR\@). Related grants contribute directly and indirectly to the success of CIGR, including NSF OCI-0721915, NSF -OCI-0725070, NSF OCI-0905046, and NSF OCI 0941653, and NSF -AST-0855535. Results presented in this article were obtained through +OCI-0725070, NSF OCI-0905046, and NSF OCI 0941653, NSF +AST-0855535, and NASA 08-ATFP08-0093. Results presented in this article were obtained through computations on the Louisiana Optical Network Initiative under allocation loni\_cactus05 and loni\_numrel06, as well as on the NSF -Teragrid under allocation TG-MCA02N014, DOE repository m152, HLRB at +Teragrid under allocations TG-MCA02N014 and TG-PHY060027N, DOE repository m152, HLRB at the LRZ, and Compute Canada project cfz-411-aa. \section*{References}

1 0

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 177)
by jfaber＠einsteintoolkit.org 07 Nov '11

07 Nov '11

User: jfaber Date: 2011/11/07 10:25 PM Modified: / ET.tex Log: Beginning editing the Examples section. File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +25 -23 =================================================================== --- ET.tex 2011-11-08 03:40:35 UTC (rev 176) +++ ET.tex 2011-11-08 04:25:03 UTC (rev 177) @@ -2282,44 +2282,45 @@ To demonstrate the properties of the code and its capabilities, we have used it to simulate common astrophysical configurations of interest. Given the community-oriented direction of the project, the parameter files required to launch these simulations and a host of others are included and documented in the code releases, along with the datafiles produced by a representative set of simulation parameters to allow for code validation and confirmation of correct code performance on new platforms and architetures. As part of the internal validation process, nightly builds are checked against a set of benchmarks to ensure that consistent results are generated with the inclusion of all new commits to the code. +The performance of the Toolkit for vacuum configurations is demonstrated through evolutions of single, rotating BHs and the merger of binary black hole configurations (sections~\ref{sec:1bh-example} and \ref{sec:bbh-example}, respectively). Linear oscillations about equilibirum for an isolated NS are discussed in section~\ref{sec:tov_oscillations}, and the collapse of a NS to a BH, including dynamical formation of a horizon, in section~\ref{sec:collapse_example}. Finally, to show a less traditional application of the code, we show its ability to perform cosmological simulations by evolving a Kasner spacetime (see section~\ref{sec:cosmology}). \subsection{Spinning BH\pages{2 Peter}} +\label{sec:1bh-example} As a first example we perform simulations of a single distorted rotating BH. We use \codename{TwoPunctures} to set up initial data for a single puncture of mass $M_{\mathrm{bh}}=1$ and dimensionless spin parameter $a = S_{\mathrm{bh}}/M_{\mathrm{bh}}^2 = 0.7$. Evolution of the data is performed by \codename{McLachlan}, apparent horizon finding by -\codename{AHFinderDirect}, gravitational wave extraction by +\codename{AHFinderDirect}, and gravitational wave extraction by \codename{WeylScal4} and \codename{Multipole}. Additional analysis of the -horizons is done by \codename{QuasiLocalMeasures}. The runs where performed -using fixed mesh refinement provided by \codename{Carpet} using 8 levels +horizons is done by \codename{QuasiLocalMeasures}. The runs were performed +with fixed mesh refinement provided by \codename{Carpet}, using 8 levels of refinement on a quadrant grid (symmetries provided by \codename{ReflectionSymmetry} and \codename{RotatingSymmetry180}). The outer -boundaries where placed at $R=256M$. We performed runs at 3 different -resolutions. The low resolution was $0.024M (3.072M)$, medium was -$0.016M (2.048M)$ and high was $0.012M (1.536M)$ where the first number is the -resolution on the finest grid and the second number in parenthesis is the -resolution on the coarsest grid. The runs where performed using the tapering -evolution scheme in \codename{Carpet} in order to avoid interpolation in +boundaries were placed at $R=256M$. We performed runs at 3 different +resolutions: the low resolution was $0.024M (3.072M)$, medium was +$0.016M (2.048M)$ and high was $0.012M (1.536M)$, where the numbers refer to the +resolution on the finest (coarsest) grid. The runs where performed using the tapering +evolution scheme in \codename{Carpet} to avoid interpolation in time during prolongation. The initial data corresponds to a rotating BH -perturbed by a Brill wave and as such has a non-zero -gravitational wave content. We evolved using 4th order finite differencing from -$T=0M$ until the BH had settled down to a stationary state at $T=120M$. +perturbed by a Brill wave and, as such, has a non-zero +gravitational wave content. We evolved the BH using 4th-order finite differencing from +$T=0M$ until it had settled down to a stationary state at $T=120M$. Figure~\ref{fig:kerr_waves} shows the $\ell =2, m=0$ mode of $r\Psi_4$ -extracted at $R=30M$ and its convergence. +extracted at $R=30M$, and its numerical convergence. \begin{figure} \includegraphics[width=0.33\textwidth,angle=-90]{examples/kerr/figs/waves} \includegraphics[width=0.33\textwidth,angle=-90]{examples/kerr/figs/waves_conv} - \caption{The right plot shows the extracted $\ell =2, m=0$ mode of $\Psi_4$ - as function of time from the high resolution run. The extraction was + \caption{The extracted $\ell =2, m=0$ mode of $\Psi_4$ + as function of time from the high resolution run (left plot). The extraction was done at $R=30M$. Shown is both the real (solid red curve) and the - imaginary (dashed green curve) part of the waveform. The left plot - shows for the real part of the $\ell =2, m=0$ waveforms the + imaginary (dashed green curve) part of the waveform. On the right, we + show the difference between the medium and low resolution runs (solid red - curve), the difference between the high and medium resolution runs - (long dashed green curve) as well as the scaled (for 4th order - convergence) difference between the medium and low resolution runs - (short dashed blue curve).} + curve), between the high and medium resolution runs + (long dashed green curve), and the scaled difference (for 4th order + convergence) between the medium and low resolution runs + (short dashed blue curve) for the real part of the $\ell =2, m=0$ waveforms.} \label{fig:kerr_waves} \end{figure} In the left plot the red (solid) curve is the real part and the green (dashed) @@ -2635,7 +2636,7 @@ \end{figure} \subsection{Linear oscillations of TOV stars\pages{2 Frank}} -\label{sec:tov_oscilations} +\label{sec:tov_oscillations} The examples in the previous subsections did not include the evolution of matter within a relativistic spacetime. One interesting test of a coupled matter-spacetime evolution is to measure the eigenfrequencies of a stable TOV @@ -2737,6 +2738,7 @@ \end{figure} \subsection{Collapse\pages{2 Christian, Roland}} +\label{sec:collapse_example} The previous examples dealt either with a preexisting black hole (BBH) or with a smooth singularity free spacetime (TOV oscillations). The evolution codes in the toolkit however are @@ -2822,7 +2824,7 @@ factor. -\subsection{Cosmology} +\subsection{Cosmology}\label{sec:cosmology} The Einstein Toolkit is not only designed to evolve compact-object spacetimes, but it is also capable of solving the initial-value problem for spacetimes with radically different topology and global

1 0

[svn:einsteintoolkit] manifest/trunk/ (Rev. 124)
by jfaber＠einsteintoolkit.org 07 Nov '11

07 Nov '11

User: jfaber Date: 2011/11/07 09:41 PM Modified: /trunk/ einsteintoolkit.bib Log: Fixed a few refs for consistency File Changes: Directory: /trunk/ ================== File [modified]: einsteintoolkit.bib Delta lines: +13 -17 =================================================================== --- trunk/einsteintoolkit.bib 2011-11-08 02:58:13 UTC (rev 123) +++ trunk/einsteintoolkit.bib 2011-11-08 03:41:24 UTC (rev 124) @@ -98,12 +98,11 @@ E.", title = "{Genesis: a high resolution code for 3-D relativistic hydrodynamics}", - journal = "Astrophys.J.Suppl.", + journal = "Astrophys. J. Suppl.", volume = "122", pages = "151", doi = "10.1086/313214", year = "1999", - note = "* Brief entry *", eprint = "astro-ph/9903352", archivePrefix = "arXiv", primaryClass = "astro-ph", @@ -810,7 +809,7 @@ journal = "Rev.Mod.Phys.", volume = "82", doi = "10.1103/RevModPhys.82.3069", - year = "19??", + year = "2010", eprint = "1010.5260", archivePrefix = "arXiv", primaryClass = "gr-qc", @@ -897,7 +896,7 @@ Hans-Thomas and Marek, Andreas and Mueller, Ewald", title = "{Generic Gravitational Wave Signals from the Collapse of Rotating Stellar Cores}", - journal = "Phys.Rev.Lett.", + journal = "Phys. Rev. Lett.", volume = "98", pages = "251101", doi = "10.1103/PhysRevLett.98.251101", @@ -1117,7 +1116,6 @@ pages = {7}, volume = {11}, keywords = {Hydrodynamics, Relativistic hydrodynamics, Numerical relativity, Magnetohydrodynamics}, - url = {http://www.livingreviews.org/lrr-2008-7} } @ARTICLE{Font:2011aa, @@ -1280,7 +1278,7 @@ @article{Gowdy:1971jh, author = "Gowdy, R.H.", title = "{Gravitational waves in closed universes}", - journal = "Phys.Rev.Lett.", + journal = "Phys. Rev. Lett.", volume = "27", pages = "826-829", doi = "10.1103/PhysRevLett.27.826", @@ -1617,7 +1615,6 @@ @ARTICLE{Lechner:2004cs, author = {{Lechner}, C. and {Alic}, D. and {Husa}, S.}, title = "{From Tensor Equations to Numerical Code -- Computer Algebra Tools for Numerical Relativity}", - journal = {eprint arXiv:cs/0411063}, eprint = {arXiv:cs/0411063}, keywords = {Computer Science - Symbolic Computation}, year = 2004, @@ -1643,7 +1640,7 @@ author = "L{\"o}ffler, Frank and Rezzolla, Luciano and Ansorg, Marcus", title = "{Numerical evolutions of a black hole-neutron star system in full general relativity: Head-on collision}", - journal = "Phys.Rev.", + journal = "Phys. Rev. ", volume = "D74", pages = "104018", doi = "10.1103/PhysRevD.74.104018", @@ -1688,7 +1685,7 @@ title = "{Momentum flow in black-hole binaries. II. Numerical simulations of equal-mass, head-on mergers with antiparallel spins}", - journal = "Phys.Rev.", + journal = "Phys. Rev. ", volume = "D82", pages = "064031", doi = "10.1103/PhysRevD.82.064031", @@ -1728,7 +1725,7 @@ @Article{Marti:1999wi, author = "Mart{\'i}, Jos{\'e} M. and M{\"u}ller, Ewald", title = "{Numerical hydrodynamics in special relativity}", - journal = "Living Rev. Rel.", + journal = "Living Reviews in Relativity", volume = 2, year = 1999, pages = 3, @@ -1758,7 +1755,7 @@ author = "Muller, Doreen and Bruegmann, Bernd", title = "{Toward a dynamical shift condition for unequal mass black hole binary simulations}", - journal = "Class.Quant.Grav.", + journal = "Class. Quant. Grav.", volume = "27", pages = "114008", doi = "10.1088/0264-9381/27/11/114008", @@ -1775,7 +1772,7 @@ and Centrella, Joan M.", title = "{Stable three level leapfrog integration in numerical relativity}", - journal = "Phys.Rev.", + journal = "Phys. Rev. ", volume = "D58", pages = "064022", doi = "10.1103/PhysRevD.58.064022", @@ -1797,12 +1794,11 @@ author = "O'Connor, Evan and Ott, Christian D.", title = "{A New Open-Source Code for Spherically-Symmetric Stellar Collapse to Neutron Stars and Black Holes}", - journal = "Class.Quant.Grav.", + journal = "Class. Quant. Grav.", volume = "27", pages = "114103", doi = "10.1088/0264-9381/27/11/114103", year = "2010", - note = "* Brief entry *", eprint = "0912.2393", archivePrefix = "arXiv", primaryClass = "astro-ph.HE", @@ -2140,7 +2136,7 @@ author = "Shibata, Masaru and Nakamura, Takashi", title = "{Evolution of three-dimensional gravitational waves: Harmonic slicing case}", - journal = "Phys.Rev.", + journal = "Phys. Rev. ", volume = "D52", pages = "5428-5444", doi = "10.1103/PhysRevD.52.5428", @@ -2217,7 +2213,7 @@ author = "Shibata, Masaru and Taniguchi, Keisuke and Uryu, Koji", title = "{Merger of binary neutron stars of unequal mass in full general relativity}", - journal = "Phys.Rev.", + journal = "Phys. Rev. ", volume = "D68", pages = "084020", doi = "10.1103/PhysRevD.68.084020", @@ -2653,7 +2649,7 @@ author = "Yoshida, Shin'ichirou and Eriguchi, Yoshiharu", title = "{Quasiradial modes of rotating stars in general relativity}", - journal = "Mon.Not.Roy.Astron.Soc.", + journal = "Mon. Not. Roy. Astron. Soc.", volume = "322", pages = "389", doi = "10.1046/j.1365-8711.2001.04115.x",

1 0

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 176)
by jfaber＠einsteintoolkit.org 07 Nov '11

07 Nov '11

User: jfaber Date: 2011/11/07 09:40 PM Modified: / ET.tex Log: Added addresses, and beginning for results section File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +12 -11 =================================================================== --- ET.tex 2011-11-08 02:58:36 UTC (rev 175) +++ ET.tex 2011-11-08 03:40:35 UTC (rev 176) @@ -97,8 +97,8 @@ \address{$^1$ Center for Computation \& Technology, Louisiana State University, Baton Rouge, LA, USA} \address{$^2$ Center for Computational Relativity and Gravitation, School of Mathematical Sciences, Rochester Institute of Technology, Rochester, NY, USA} -\address{$^3$ AEI \todo{complete}} -\address{$^4$ GTech \todo{complete}} +\address{$^3$ Max-Planck-Institut f\"{u}r Gravitationsphysik, Albert-Einstein-Institut, Golm, Germany} +\address{$^4$ Center for Relativistic Astrophysics, School of Physics, Georgia Institute of Technology, Atlanta, GA, USA} \address{$^5$ TAPIR, California Institute of Technology, Pasadena, CA, USA} \address{$^6$ Perimeter Institute for Theoretical Physics, Waterloo, ON, Canada} @@ -677,7 +677,7 @@ from the following features added to Kranc after the codes themselves were written: (i) OpenMP parallelization support, necessary for efficient use of modern multi-core processors; (ii) support for -multipatch domains with the Llama\cite{Pollney:2009yz} code; (iii) automatic generation of +multipatch domains with the Llama~\cite{Pollney:2009yz} code; (iii) automatic generation of vectorized code, where the equations are evaluated simultaneously by the processor for two grid points at the same time, and (iv) common sub-expression elimination, and various other optimization strategies. @@ -693,11 +693,9 @@ Some of these modules provide common definitions and conventions (see section~\ref{sec:base_modules}). Others provide initial data (see section~\ref{sec:initial_data}), which may -be evolved using the different evolution methods described in -sections~\ref{sec:Kevol} and~\ref{sec:GRHydro}. -Simulations including -matter require their own evolution techniques (see section~\ref{sec:GRHydro}) -and a description of the matter properties, encoded in equations +be evolved using the different evolution methods for vacuum and matter configurations described in +sections~\ref{sec:Kevol} and~\ref{sec:GRHydro}, respectively. +The thermodynamic properties of fluids are encoded in equations of state (see section~\ref{sec:eoss}). Finally, additional quantities besides those being directly evolved are often interesting for a detailed analysis of the simulation's results. @@ -1021,7 +1019,7 @@ two puncture locations. \codename{TwoPunctures} resolves this problem by performing a coordinate transformation modeled on confocal elliptical/hyperbolic coordinates, which transforms the spatial domain into a finite cube with the puncture locations mapped -to two parallel edges, as can be seen in Fig.~\ref{fig:TP_BHNS_coordinates}. +to two parallel edges, as can be seen in figure~\ref{fig:TP_BHNS_coordinates}. The coordinate transformation is \begin{eqnarray} x&=&b\frac{A^2+1}{A^2-1}\frac{2B}{1+B^2}\nonumber\\ @@ -1063,7 +1061,7 @@ Matter source terms are ideal for this split, since they are compactly supported, while extrinsic curvature source terms are spatially extended, but with sufficiently rapid falloff at large radii to yield convergent solutions. Around each object, -a set of nested spheroidal sub-domains (see Fig.~\ref{fig:Lorene_coordinates}) is constructed extending to cover all +a set of nested spheroidal sub-domains (see figure~\ref{fig:Lorene_coordinates}) is constructed extending to cover all of space, with the outermost domain incorporating a compactification to allow it to extend to spatial infinity. Within each of the nested sub-domains, fields are decomposed into Chebyshev modes radially and spherical harmonics @@ -2045,7 +2043,7 @@ of a projection onto spin-weighted spherical harmonics ${}_s Y_{\ell m}$. %\todo{CCE} -% JF: are we discussing CCE in this paper? +% JF: are we discussing CCE in this paper? WE say not in the conclusions... \subsubsection{Object tracking} \label{sec:object-tracking} @@ -2281,6 +2279,9 @@ \section{Examples\pages{0.5 Frank}} \todo{Will be written once we know which examples we have} +To demonstrate the properties of the code and its capabilities, we have used it to simulate common astrophysical configurations of interest. Given the community-oriented direction of the project, the parameter files required to launch these simulations and a host of others are included and documented in the code releases, along with the datafiles produced by a representative set of simulation parameters to allow for code validation and confirmation of correct code performance on new platforms and architetures. As part of the internal validation process, +nightly builds are checked against a set of benchmarks to ensure that consistent results are generated with the inclusion of all new commits to the code. + \subsection{Spinning BH\pages{2 Peter}} As a first example we perform simulations of a single distorted rotating BH. We use \codename{TwoPunctures} to set up initial data for a single

1 0

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 175)
by cott＠tapir.caltech.edu 07 Nov '11

07 Nov '11

User: cott Date: 2011/11/07 08:58 PM Modified: / ET.tex Log: * typo File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +1 -1 =================================================================== --- ET.tex 2011-11-08 02:57:52 UTC (rev 174) +++ ET.tex 2011-11-08 02:58:36 UTC (rev 175) @@ -2908,7 +2908,7 @@ Besides new additions of physics modules, existing techniques require improvement. One example is the need for the gauge invariant extraction of gravitational waves from simulation spacetimes as realized -by the Cauche Characteristic Extraction (CCE) technique recently studied +by the Cauchy Characteristic Extraction (CCE) technique recently studied in \cite{Babiuc:11,Reisswig:2010cd,Reisswig:2011a}. The authors of one such CCE code \cite{Babiuc:11} have agreed to make their work available to the whole community by integrating their CCE routines into the Einstein Tookit

1 0

[svn:einsteintoolkit] manifest/trunk/ (Rev. 123)
by cott＠tapir.caltech.edu 07 Nov '11

07 Nov '11

User: cott Date: 2011/11/07 08:58 PM Modified: /trunk/ einsteintoolkit.bib Log: * add few refs needed in the ET paper File Changes: Directory: /trunk/ ================== File [modified]: einsteintoolkit.bib Delta lines: +55 -0 =================================================================== --- trunk/einsteintoolkit.bib 2011-11-07 19:14:00 UTC (rev 122) +++ trunk/einsteintoolkit.bib 2011-11-08 02:58:13 UTC (rev 123) @@ -261,6 +261,24 @@ ##B ############################################################################ +@ARTICLE{Babiuc:11, + author = {{Babiuc}, M.~C. and {Szil{\'a}gyi}, B. and {Winicour}, J. and + {Zlochower}, Y.}, + title = "{Characteristic extraction tool for gravitational waveforms}", + journal = {Phys Rev. D.}, +archivePrefix = "arXiv", + eprint = {arXiv:1011.4223}, + year = 2011, + month = aug, + volume = 84, + number = 4, + eid = {044057}, + pages = {044057}, + doi = {10.1103/PhysRevD.84.044057}, + adsurl = {http://adsabs.harvard.edu/abs/2011PhRvD..84d4057B}, +} + + @Article{Baiotti:2004wn, requested-for ="EinsteinEvolve/GRHydro", author = "Luca Baiotti and Ian Hawke and Pedro J. Montero and @@ -1936,6 +1954,26 @@ year = 1957, } +@ARTICLE{Reisswig:2011a, + author = {{Reisswig}, C. and {Bishop}, N.~T. and {Pollney}, D. and {Szil{\'a}gyi}, B. + }, + title = "{Characteristic extraction in numerical relativity: binary black hole merger waveforms at null infinity}", + journal = {Classical and Quantum Gravity}, +archivePrefix = "arXiv", + eprint = {0912.1285}, + primaryClass = "gr-qc", + year = 2010, + month = apr, + volume = 27, + number = 7, + pages = {075014}, + doi = {10.1088/0264-9381/27/7/075014}, + adsurl = {http://adsabs.harvard.edu/abs/2010CQGra..27g5014R}, + adsnote = {Provided by the SAO/NASA Astrophysics Data System} +} + + + @article{Reisswig:2010cd, author = "Reisswig, Christian and Ott, Christian D. and Sperhake, Ulrich and Schnetter, Erik", @@ -2461,6 +2499,21 @@ SLACcitation = "%%CITATION = PHRVA,55,364;%%" } +@ARTICLE{Toth:00, + author = {{T{\'o}th}, G.}, + title = "{The ${\nabla}.B=0$ Constraint in Shock-Capturing Magnetohydrodynamics Codes}", + journal = {Journal of Computational Physics}, + year = 2000, + month = jul, + volume = 161, + pages = {605-652}, + doi = {10.1006/jcph.2000.6519}, + adsurl = {http://adsabs.harvard.edu/abs/2000JCoPh.161..605T}, + adsnote = {Provided by the SAO/NASA Astrophysics Data System} +} + + + ##U ############################################################################ ##V ############################################################################ @@ -2665,3 +2718,5 @@ year = 2005, eprint = "arXiv:gr-qc/0505055", } + +

1 0

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 174)
by cott＠tapir.caltech.edu 07 Nov '11

07 Nov '11

User: cott Date: 2011/11/07 08:57 PM Modified: / ET.tex Log: * new version of the conclusions and future work File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +93 -52 =================================================================== --- ET.tex 2011-11-07 20:06:30 UTC (rev 173) +++ ET.tex 2011-11-08 02:57:52 UTC (rev 174) @@ -85,10 +85,10 @@ Eloisa Bentivegna $^3$, Tanja Bode $^4$, Peter Diener $^1$, -Roland Haas $^5$, +Roland Haas $^{5,4}$, Ian Hinder $^3$, Bruno C. Mundim $^2$, -Christian D. Ott $^5$, +Christian D. Ott $^{5,1}$, Erik Schnetter $^{6,7,1}$, Gabrielle Allen $^1$, Manuela Campanelli $^2$ @@ -99,7 +99,7 @@ \address{$^2$ Center for Computational Relativity and Gravitation, School of Mathematical Sciences, Rochester Institute of Technology, Rochester, NY, USA} \address{$^3$ AEI \todo{complete}} \address{$^4$ GTech \todo{complete}} -\address{$^5$ Caltech \todo{complete}} +\address{$^5$ TAPIR, California Institute of Technology, Pasadena, CA, USA} \address{$^6$ Perimeter Institute for Theoretical Physics, Waterloo, ON, Canada} \address{$^7$ Department of Physics, University of Guelph, Guelph, ON, @@ -2870,61 +2870,102 @@ \end{figure} -\section{Future Work} -In this paper we illustrated the state of the ``Einstein Toolkit'' release ``Curie'', -a collection of freely available and easy to use computational codes -for numerical relativity. However, there is room for improvement on both -the underlying infrastructure and the included physics. +\section{Conclusion and Future Work} +In this article, we described the Einstein Toolkit, a collection +of freely available and easy to use computational codes for numerical +relativity and relativistic astrophysics. The code details and example +results present in this article represent the state of the Einstein +Toolkit in its release ET\_2011\_05 ``Curie,'' released on April 21, +2011. -One of the desirable additions of physics is a proper treatment of -radiation, in particular neutrinos. Another desirable addition would be some -approximation of emission of electro-magnetic waves. Radiation transport is, -even compared to the full GRMHD problem, computationally very expensive, -especially in three dimensions. Several members of the CIGR team are involved -in the NSF project PetaCactus, which aims to explore these possibilities. +What was presented here is but a snapshot of the Einstein Toolkit's +ongoing development whose ultimate goal it is to provide an +open-source set of robust baseline codes to realistically and +reproducibly model the whole spectrum of relativistic astrophysical +phenomena including, but not limited to, isolated black holes and +neutron stars, binary black hole coalescence in vacuum and gaseous +environs, double neutron star and neutron star -- black hole mergers, +core-collapse supernovae, and gamma-ray bursts. -Besides new additions, existing techniques require to be improved. One such -example is the improvement of current wave extraction techniques within the -Einstein Toolkit to include Cauchy characteristic wave extraction techniques, -as recently studied in~\cite{???}. \todo{citation} The authors of these research codes -agreed to make their work available to the whole community within -the Einstein toolkit, which happened in the recently released version ``Maxwell'', -but which is not described here. +For this, much future work towards including proper treatments of +magnetic fields, more complex equations of state, nuclear reactions, +neutrinos, and photons will be necessary and will need to be matched +by improvements in infrastructure (e.g., more flexible AMR on general +grids) and computing hardware for the required fully coupled 3D, +multi-scale, multi-physics simulations to become reality. -A second example for an improvement of an existing method is extending the -current implementation of ideal magnetohydrodynamics to the case of non-zero -resistivity. Such implementations exist -and are described in the literature~\cite{???} \todo{cite}, but none have been provided freely -within the Einstein Toolkit until the recently released ``Maxwell'' version. -Resistive MHD is important for a number of astrophysical scenarios, one of -which is the merger of two magnetized NSs, a candidate for short gamma-ray -bursts. -\todo{reword - make clear that we describe Curie} +%%% One of the desirable additions of physics is a proper treatment of +%%% radiation, in particular neutrinos. Another desirable addition would be some +%%% approximation of emission of electro-magnetic waves. Radiation transport is, +%%% even compared to the full GRMHD problem, computationally very expensive, +%%% especially in three dimensions. -Yet another important goal is to increase the scalability of the {\tt Carpet} AMR -infrastructure. It has been shown that good scaling is limited to less than -a few thousand processes, for some of the most used simulation scenarios. -In this case, work is already in progress to eliminate this bottle-neck. -On the other hand, a production simulation is typically composed from a large -number of components, and all of them have to scale well to achieve overall -good performance. This is not an easy problem, as most of the module authors -are neither computer scientists nor had they extensive training in parallel -development and profiling techniques. Close collaboration with such experts -has been shown to be very fruitful in the past and is definitely also planned -for the foreseeable future. +%%% CDO: It is not appropriate to mention a grant like PetaCactus as an ``NSF Project''. +%%% Please leave this out. +%%% Several members of the CIGR team are involved +%%% in the NSF project PetaCactus, which aims to explore these possibilities. +Besides new additions of physics modules, existing techniques require +improvement. One example is the need for the gauge invariant +extraction of gravitational waves from simulation spacetimes as realized +by the Cauche Characteristic Extraction (CCE) technique recently studied +in \cite{Babiuc:11,Reisswig:2010cd,Reisswig:2011a}. The authors of one such +CCE code \cite{Babiuc:11} have agreed to make their work available to the +whole community by integrating their CCE routines into the Einstein Tookit +release 2011\_11 ``Maxwell,'' which will be described elsewhere. + +A second example for a much needed improvement of an existing method +is to transition to cell-centered AMR for GR hydrodynamic simulations, +which would allow for exact flux conservation across AMR interfaces +via a refluxing step that adjusts coarse and/or fine grid fluxes for +consistency (e.g., \cite{Berger:1984zza}). This is also a prerequisite +for the constrained transport method \cite{Toth:00} for ensuring the +divergence-free condition for the magnetic field in a future +implementation of GRMHD within the Einstein Toolkit. Work towards +cell-centered AMR, refluxing, and GRMHD is underway and will be +reported in a future publication. + + +%%% CDO: We have not even talked about the current MHD implementation, +%%% how can we talk about its improvement? +%%% +%%% A second example for an improvement of an existing method is extending the +%%% current implementation of ideal magnetohydrodynamics to the case of non-zero +%%% resistivity. Such implementations exist +%%% and are described in the literature~\cite{???} \todo{cite}, but none have been provided freely +%%% within the Einstein Toolkit until the recently released ``Maxwell'' version. +%%% Resistive MHD is important for a number of astrophysical scenarios, one of +%%% which is the merger of two magnetized NSs, a candidate for short gamma-ray +%%% bursts. +%%% \todo{reword - make clear that we describe Curie} + +Yet another important goal is to increase the scalability of the {\tt + Carpet} AMR infrastructure. As we have shown, good scaling is +limited to only a few thousand processes for some of the most used +simulation scenarios. Work is in progress to eliminate this +bottle-neck. On the other hand, a production simulation is typically +composed of a large number of components, and even analysis and I/O +routines have to scale well to achieve overall good performance. This +is a highly non-trivial problem, since most Einstein Toolkit physics +module authors are neither computer scientists nor have they had +extensive training in parallel development and profiling +techniques. Close collaboration with experts in these topics has been +fruitful in the past and will be absolutely necessary for the +optimization of Einstein Toolkit codes for execution on the upcoming +generation of true petascale supercomputers on which typical compute jobs +are expected to be running on 100,000 and more compute cores. + \section*{Acknowledgments\pages{0.5 All}} -\todo{Frank, ADD} -The Einstein Toolkit is directly supported by the National Science Foundation -under the grant numbers 0903973/0903782/0904015 (CIGR\@). -Related grants contribute directly and indirectly to the success of CIGR, -including NSF OCI 0721915 ``Alpaca'', NSF OCI 0725070 -``Blue Waters'', NSF OCI 0905046 ``PetaCactus'', and -NSF OCI 0941653 ``PRAC''. Results presented -in this article were obtained through computations on the Louisiana Optical -Network Initiative under allocation loni\_cactus05 and loni\_numrel06, as well -as on the NSF Teragrid under allocation TG-MCA02N014, DOE repository m152, -HLRB at the LRZ, and Compute Canada project cfz-411-aa. +\todo{Frank, ADD} The Einstein Toolkit is directly supported by the +National Science Foundation under the grant numbers +0903973/0903782/0904015 (CIGR\@). Related grants contribute directly +and indirectly to the success of CIGR, including NSF OCI-0721915, NSF +OCI-0725070, NSF OCI-0905046, and NSF OCI 0941653, and NSF +AST-0855535. Results presented in this article were obtained through +computations on the Louisiana Optical Network Initiative under +allocation loni\_cactus05 and loni\_numrel06, as well as on the NSF +Teragrid under allocation TG-MCA02N014, DOE repository m152, HLRB at +the LRZ, and Compute Canada project cfz-411-aa. \section*{References}

1 0

[svn:einsteintoolkit] incoming/PITTNullCode/SphericalHarmonicReconASCII/ (Rev. 57)
by reisswig＠tapir.caltech.edu 07 Nov '11

07 Nov '11

User: reisswig Date: 2011/11/07 06:03 PM Modified: /PITTNullCode/SphericalHarmonicReconASCII/ configuration.ccl, param.ccl /PITTNullCode/SphericalHarmonicReconASCII/src/ sph_database.cc, sph_database.hh, startup.cc Log: Add capability of hdf5. File Changes: Directory: /PITTNullCode/SphericalHarmonicReconASCII/ ===================================================== File [modified]: configuration.ccl Delta lines: +1 -1 =================================================================== --- PITTNullCode/SphericalHarmonicReconASCII/configuration.ccl 2011-09-04 20:16:09 UTC (rev 56) +++ PITTNullCode/SphericalHarmonicReconASCII/configuration.ccl 2011-11-08 00:03:19 UTC (rev 57) @@ -1 +1 @@ -REQUIRES +REQUIRES HDF5 File [modified]: param.ccl Delta lines: +3 -2 =================================================================== --- PITTNullCode/SphericalHarmonicReconASCII/param.ccl 2011-09-04 20:16:09 UTC (rev 56) +++ PITTNullCode/SphericalHarmonicReconASCII/param.ccl 2011-11-08 00:03:19 UTC (rev 57) @@ -5,8 +5,9 @@ KEYWORD format "Boundary data file format" { - ASCII :: "standard ASCII format: lm-modes and spheres are stored in a 2d array (gnuplot style) for a given timestep" - DAT :: "all variables and lm-modes are stored in one row for a given timestep" + ASCII :: "standard ASCII format: lm-modes and spheres are stored in a 2d array (gnuplot style) for a given timestep" + DAT :: "optimized ASCII format: all variables and lm-modes are stored in one row for a given timestep" + SpEC-H5 :: "SpEC-HDF5 format: Data is stored in HDF5 files grouped according to SpEC output." } "ASCII" CCTK_INT sphere_number "the sphere number (if multiple spheres are present in one file) that corresponds to the worldtube radius NullSHRExtract::cr" Directory: /PITTNullCode/SphericalHarmonicReconASCII/src/ ========================================================= File [modified]: sph_database.cc Delta lines: +227 -0 =================================================================== --- PITTNullCode/SphericalHarmonicReconASCII/src/sph_database.cc 2011-09-04 20:16:09 UTC (rev 56) +++ PITTNullCode/SphericalHarmonicReconASCII/src/sph_database.cc 2011-11-08 00:03:19 UTC (rev 57) @@ -855,7 +855,234 @@ } + + + + + + + + + + + + + +//------------------------------------------------------------------------------------------- +// +// +// SpEC-H5 format goes here........... +// +// +//------------------------------------------------------------------------------------------ + + + + +const string SPH_db_SpEC_H5::time_attrib_name = "Time"; +const string SPH_db_SpEC_H5::step_basename = "Step"; + + // table of varibale names: this maps a column number to a group. + // The first entry per varibale is the main group, the second is a possible subgroup, e.g. + // /g/Step00001/xx + // for SpEC +const string SPH_db_SpEC_H5::varname_table[30][2] = + { { "Lapse", "scalar"}, + { "Shift", "x" }, { "Shift", "y" }, { "Shift", "z" }, + { "g", "xx" }, { "g", "xy" }, { "g", "xz" }, + { "g", "yy" }, { "g", "yz" }, + { "g", "zz" }, + + { "DrLapse", "scalar"}, + { "DrShift", "x" }, { "DrShift", "y" }, { "DrShift", "z" }, + { "Drg", "xx" }, { "Drg", "xy" }, { "Drg", "xz" }, + { "Drg", "yy" }, { "Drg", "yz" }, + { "Drg", "zz" }, + + { "DtLapse", "scalar"}, + { "DtShift", "x" }, { "DtShift", "y" }, { "DtShift", "z" }, + { "Dtg", "xx" }, { "Dtg", "xy" }, { "Dtg", "xz" }, + { "Dtg", "yy" }, { "Dtg", "yz" }, + { "Dtg", "zz" } }; + + +SPH_db_SpEC_H5::SPH_db_SpEC_H5(const string& fname_, + const int cached_timesteps) + : SPH_database(cached_timesteps), _fname(fname_) +{ + // open file + file = H5Fopen(_fname.c_str(), H5F_ACC_RDONLY, H5P_DEFAULT); + + scan(); } + +SPH_db_SpEC_H5::~SPH_db_SpEC_H5() +{ + // close file + H5Fclose(file); +} + +int SPH_db_SpEC_H5::remove_non_monotonic_steps(const int k) +{ + // go backwards from k and remove until we get monotonicity + const double time = steps[k].time; + const int size = steps.size(); + int c = 1; + for (int i=k-1; i >= 0; --i) { + if (time > steps[i].time) { + break; + } + ++c; + } + + steps.erase(steps.begin()+k-c, steps.begin()+k); + _times.erase(_times.begin()+k-c, _times.begin()+k); + _iterations.erase(_iterations.begin()+k-c, _iterations.begin()+k); + + // return the number of steps that were removed + return size-steps.size(); +} + + +void SPH_db_SpEC_H5::scan() +{ + + // scan datasets + scan_HDF5(); + + + // check continuity and delta_t of timesteps + if (_times.size() > 1) + { + cout << "SPH_db_SpEC_H5 " << _fname << ": t_0 = " << _times[0] << ", t_final = " << _times.back() << endl; + _delta_t = 0; + for (int i=1; i < _times.size(); ++i) + { + if (fabs(_times[i] - _times[i-1]) < 1e-8 || _times[i] < _times[i-1]) { + ostringstream str; + str << "Removing non-monotonic timesteps: time[" << i-1 << "] = " << _times[i-1] << ", time[" << i << "] = " << _times[i]; + CCTK_WARN(1, str.str().c_str()); + i -= remove_non_monotonic_steps(i); + continue; + } + if (fabs(_times[i] - _times[i-1] - _delta_t) >= 1e-8) + { + _delta_t = _times[i]-_times[i-1]; + cout << "SPH_db_SpEC_H5 " << _fname << ": t = " << _times[i-1] << ", delta_t = " << _delta_t << endl; + } + } + // set _delta_t to correspond to initial delta_t + _delta_t = _times[1]-_times[0]; + } + +} + + + + +void SPH_db_SpEC_H5::read(const int timestep, const int varno, + vector<vector<vector<CCTK_COMPLEX> > >& coeff) const +{ + if (timestep >= steps.size()) { + CCTK_WARN(0, "Requested timestep not in worldtube data file. Stopping!"); + } + + hid_t dataset = H5Dopen(file, (varname_table[varno][0]+"/"+steps[timestep].name+"/"+varname_table[varno][1]).c_str());//, H5P_DEFAULT); + vector<double> buffer((_lmax+1)*(_lmax+1)*2); + H5Dread(dataset, H5T_NATIVE_DOUBLE, H5S_ALL, H5S_ALL, H5P_DEFAULT, &buffer.front()); + H5Dclose(dataset); + + int c = 0; + for (int l=0; l <= _lmax; ++l) { + for (int m=l; m >= -l; --m, ++c) { + //coeff[0][l][m+l] = CCTK_Cmplx(buffer[c], buffer[c + buffer.size()/2]); + coeff[0][l][m+l] = CCTK_Cmplx(buffer[2*c], buffer[2*c + 1]); + } + } +} + + + +herr_t SPH_db_SpEC_H5::H5iter(hid_t loc_id, const char* name, const H5L_info_t* info, void* operator_data) +{ + SPH_db_SpEC_H5* SpEC_H5 = (SPH_db_SpEC_H5*) operator_data; + + // Datasets are ignored! We only iterate over groups + switch (info->type) { + case H5O_TYPE_GROUP: { + const string gname = name; + // we have found a timstep group if the group contains the "step_basename" substring + if (gname.find(step_basename) != string::npos) { + // get time attribute + double time; + hid_t group = H5Gopen(loc_id, name); + hid_t attribute = H5Aopen(group, time_attrib_name.c_str(), H5P_DEFAULT); + H5Aread(attribute, H5T_NATIVE_DOUBLE, &time); + H5Aclose(attribute); + H5Gclose(group); + + SpEC_H5->steps.push_back(timestep_t(name, time)); + } + break; + } + case H5O_TYPE_DATASET: + // ignore datasets + return 0; + break; + default: + break; + } + + return 0; +} + + + +void SPH_db_SpEC_H5::scan_HDF5() +{ + // pick one variable and browse through entire file + // to find out how many (and what) cycles and times there are + // (assuming that all variables are present at the same timesteps) + herr_t status = H5Literate_by_name (file, varname_table[0][0].c_str(), H5_INDEX_NAME, H5_ITER_NATIVE, NULL, H5iter, this, H5P_DEFAULT); + // sort 'steps' vector + //sort(steps.begin(), steps.end()); + _iterations.resize(steps.size()); + _times.resize(steps.size()); + for (int i=0; i < steps.size(); ++i) { + //cout << steps[i].name << " " << steps[i].time << endl; + _iterations[i] = i; + _times[i] = steps[i].time; + } + + // get lmax from first variable and timestep + hid_t dataset = H5Dopen(file, (varname_table[0][0]+"/"+steps[0].name+"/"+varname_table[0][1]).c_str()); //, H5P_DEFAULT); + //cout << (varname_table[0][0]+"/"+steps[0].name+"/"+varname_table[0][1]) << endl; + hid_t dataspace = H5Dget_space(dataset); + hsize_t size = H5Sget_simple_extent_npoints(dataspace); + H5Sclose(dataspace); + H5Dclose(dataset); + + _lmax = sqrt(size/2)-1; + _n_variables = 30; + _n_spheres = 1; + + // cross-check that all expected variables are present + + + // display some info + cout << "SPH_db_SpEC_H5 " << _fname << ": n_spheres = " << _n_spheres << endl; + cout << "SPH_db_SpEC_H5 " << _fname << ": n_variables = " << _n_variables << endl; + cout << "SPH_db_SpEC_H5 " << _fname << ": lmax = " << _lmax << endl; + cout << "SPH_db_SpEC_H5 " << _fname << ": timesteps = " << _iterations.size() << endl; +} + + + + + + + +} File [modified]: sph_database.hh Delta lines: +63 -0 =================================================================== --- PITTNullCode/SphericalHarmonicReconASCII/src/sph_database.hh 2011-09-04 20:16:09 UTC (rev 56) +++ PITTNullCode/SphericalHarmonicReconASCII/src/sph_database.hh 2011-11-08 00:03:19 UTC (rev 57) @@ -18,10 +18,14 @@ #include "mpi.h" +#define H5_USE_16_API 1 +#include "hdf5.h" #ifndef _SPH_database_ #define _SPH_database_ + + namespace SHR { using namespace std; @@ -276,7 +280,66 @@ + /** + A SpEC-HDF5 database +*/ +class SPH_db_SpEC_H5 : public SPH_database +{ + public : + SPH_db_SpEC_H5(const string& fname_, + const int cached_timesteps); + + virtual ~SPH_db_SpEC_H5(); + + // scans the input file + virtual void scan(); + + // read all modes on all extraction spheres for a given timestep and variable number into array + virtual void read(const int timestep, const int varno, + vector<vector<vector<CCTK_COMPLEX> > >& coeff) const; + + private : + + int remove_non_monotonic_steps(const int k); + void scan_HDF5(); + static herr_t H5iter(hid_t loc_id, const char* name, const H5L_info_t* info, void* operator_data); + + + string _fname; + + // the hdf5 file handle + hid_t file; + + static const string time_attrib_name; + static const string step_basename; + + // table of varibale names: this maps a column number to a group. + // The first entry per varibale is the main group, the second is a possible subgroup, e.g. + // /g/Step00001/xx + // for SpEC + static const string varname_table[30][2]; + + struct timestep_t + { + string name; + double time; + timestep_t(const string& name_, const double time_) + : name(name_), time(time_) + {} + bool operator<(const timestep_t& rhs) const { return time < rhs.time; } + }; + + /// the cylces and names of the steps contained in the file + vector<timestep_t> steps; + +}; + + + + + +/** This class represents a set of spherical coefficients for one variable and over all extraction spheres. It can read-in the coefficients from a database and reconstruct the angular dependence by recomposing with the basis-functions, i.e. the sYlm. File [modified]: startup.cc Delta lines: +2 -0 =================================================================== --- PITTNullCode/SphericalHarmonicReconASCII/src/startup.cc 2011-09-04 20:16:09 UTC (rev 56) +++ PITTNullCode/SphericalHarmonicReconASCII/src/startup.cc 2011-11-08 00:03:19 UTC (rev 57) @@ -122,6 +122,8 @@ SHR::db[i] = new SHR::SPH_db_ASCII(pathname+fnames[i], cached_timesteps+1, column_time, column_iteration, column_radius); if (CCTK_EQUALS(format, "DAT")) SHR::db[i] = new SHR::SPH_db_DAT(pathname+fnames[i], cached_timesteps+1, lmaxInFile, column_time, column_iteration, column_radius, column_lmax, column_n_variables, column_data); + if (CCTK_EQUALS(format, "SpEC-H5")) + SHR::db[i] = new SHR::SPH_db_SpEC_H5(pathname+fnames[i], cached_timesteps+1); // check if timestep-size found in file matches characteristic timestep! if (fabs(SHR::db[i]->delta_t()-CCTK_DELTA_TIME) > 1e-8)

1 0

[svn:einsteintoolkit] Paper_EinsteinToolkit_2010/ (Rev. 173)
by knarf＠cct.lsu.edu 07 Nov '11

07 Nov '11

User: knarf Date: 2011/11/07 02:06 PM Modified: / ET.tex Log: remove todo, sounds fine File Changes: Directory: / ============ File [modified]: ET.tex Delta lines: +15 -2 =================================================================== --- ET.tex 2011-11-07 19:58:45 UTC (rev 172) +++ ET.tex 2011-11-07 20:06:30 UTC (rev 173) @@ -120,8 +120,21 @@ % to be used \begin{abstract} -We describe the Einstein Toolkit, a community-driven, freely accessible computational infrastructure intended for use in numerical relativity, relativistic astrophysics, and other applications. The Toolkit, developed by a collaboration involving researchers from several institutions in the United States and others around the world, combines all of the components needed to simulate astrophysical objects including black holes, compact objects, and collapsing stars, as well as a full suite of analysis tools. It is based on commonly used packages, such as the Cactus Framework for HPC computing and the Carpet adaptive mesh refinement driver, but is under continuous development and contains many new code components that have been publicly released for the first time. We discuss the motivation behind the release of the toolkit, the philosophy underlying its development, and the goals of the project. A brief summary of numerical techniques is included, as are results of numerical test covering a variety of sample astrophysical problems. We conclude with a discussion of future directions in which the toolkit is likely to be expanded. -\todo{ADD more, Frank} +We describe the Einstein Toolkit, a community-driven, freely accessible +computational infrastructure intended for use in numerical relativity, +relativistic astrophysics, and other applications. The Toolkit, developed by a +collaboration involving researchers from several institutions in the United +States and others around the world, combines all of the components needed to +simulate astrophysical objects including black holes, compact objects, and +collapsing stars, as well as a full suite of analysis tools. It is based on +commonly used packages, such as the Cactus Framework for HPC computing and the +Carpet adaptive mesh refinement driver, but is under continuous development and +contains many new code components that have been publicly released for the +first time. We discuss the motivation behind the release of the toolkit, the +philosophy underlying its development, and the goals of the project. A brief +summary of numerical techniques is included, as are results of numerical test +covering a variety of sample astrophysical problems. We conclude with a +discussion of future directions in which the toolkit is likely to be expanded. \end{abstract} \pacs{04.25.D-, 04.30.-w, 04.70.-s, 07.05.Tp, 95.75.Pq}

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Commits