Dear ET folks,
I have written a small thorn that converts HydroBase and ADMBase variables into the variables IllinoisGRMHD needs to perform a GRMHD evolution. This conversion works just fine on all variables... except the vector potential.
As you may be aware, IllinoisGRMHD requires that A_t (=\sqrt{\gamma} \Phi), A_x, A_y, and A_z each be staggered differently (a la Table I of http://arxiv.org/abs/1007.2848).
However, HydroBase does not support defining such staggered A-fields. Thus it is currently impossible to modify/create GRMHD initial data thorns so that vector potential data produced by them can be directly read in by IllinoisGRMHD. For example, if an initial data thorn only outputs the vector potential on grid vertices, then an interpolation will need to be performed to staggered gridpoints. This will introduce truncation errors that, depending on the A-field structure and needed grid resolutions, may be severe. I conclude that this is _not_ The Right Strategy. The Right Strategy would not break any existing thorn as a first step, either.
So as a first step, I believe we should modify HydroBase so that 1) A new set of four, 3-timelevel groups are defined, one for each component of the vector potential 2) The prolongation type can be specified differently for each of the four vector potential gridfunction groups, allowing for arbitrary staggerings of the single variable within each of these groups.
Variables in these groups will be read in by IllinoisGRMHD and can be used directly for evolutions.
Regarding the longer-term strategy, we could either
1) modify Carpet so that different components of Avec() (the current 3-vector potential group in HydroBase) can accept different prolongation/restriction operators (then modifying IllinoisGRMHD to accept this new vector gridfunction), or
2) just keep the four, 3-timelevel groups and phase out Avec() and Aphi altogether.
Even the latter strategy would not be difficult to implement in current publicly available thorns:
After a simple grep, I could only find a single initial data thorn that actually sets Avec, so the needed changes to public initial data thorns should be minimal. Further, I am under the impression that Avec evolutions are not yet supported in GRHydro anyway, and found only about 40 references to the Avec() gridfunctions within GRHydro, so renaming these variables to a new style would be trivial.
Please let me know your thoughts regarding this proposal. The IllinoisGRMHD code announcement paper is nearing completion, and at least having the infrastructure for this code in place by the next ET release would be great. In this regard, I have already asked Erik if he would be willing to merge the staggered prolongation/restriction code into the mainline Carpet for the next ET release. The soft freeze is Nov 10, these new features affect none of Carpet's other functionality, and I have thoroughly tested these new features with GRMHD evolutions, verifying that they work well.
Please consider the possibility of adding these small infrastructure enhancements prior to the freeze.
Sincerely,
-Zach
* * * Zachariah Etienne Assistant Professor of Mathematics West Virginia University
Whether A^i lives on the same grid points as the metric or is staggered is part of its definition. Differing definitions should be stored in different grid functions, even if this seems to lead to a duplication. Cactus used to have different definitions of the metric, and used to store them in the same grid functions. There was a (slightly complex) mechanism that defined and recorded which definition was currently active. This led to errors, as people either didn't get the mechanism for choosing the metric definition right, or didn't add the respective checks and conversions into their code.
Defining a new grid function is cheap, and we should store the different discretizations of A^i in such a way.
Currently, Carpet cannot handle different prolongation operators for different grid variables in the same group. This should probably change. (Does it with your proposed changes?) We also need to see how this affects symmetry conditions.
-erik
On Thu, Oct 30, 2014 at 1:51 PM, Zach Etienne zachetie@gmail.com wrote:
Dear ET folks,
I have written a small thorn that converts HydroBase and ADMBase variables into the variables IllinoisGRMHD needs to perform a GRMHD evolution. This conversion works just fine on all variables... except the vector potential.
As you may be aware, IllinoisGRMHD requires that A_t (=\sqrt{\gamma} \Phi), A_x, A_y, and A_z each be staggered differently (a la Table I of http://arxiv.org/abs/1007.2848).
However, HydroBase does not support defining such staggered A-fields. Thus it is currently impossible to modify/create GRMHD initial data thorns so that vector potential data produced by them can be directly read in by IllinoisGRMHD. For example, if an initial data thorn only outputs the vector potential on grid vertices, then an interpolation will need to be performed to staggered gridpoints. This will introduce truncation errors that, depending on the A-field structure and needed grid resolutions, may be severe. I conclude that this is _not_ The Right Strategy. The Right Strategy would not break any existing thorn as a first step, either.
So as a first step, I believe we should modify HydroBase so that
- A new set of four, 3-timelevel groups are defined, one for each component
of the vector potential 2) The prolongation type can be specified differently for each of the four vector potential gridfunction groups, allowing for arbitrary staggerings of the single variable within each of these groups.
Variables in these groups will be read in by IllinoisGRMHD and can be used directly for evolutions.
Regarding the longer-term strategy, we could either
- modify Carpet so that different components of Avec() (the current
3-vector potential group in HydroBase) can accept different prolongation/restriction operators (then modifying IllinoisGRMHD to accept this new vector gridfunction), or
- just keep the four, 3-timelevel groups and phase out Avec() and Aphi
altogether.
Even the latter strategy would not be difficult to implement in current publicly available thorns:
After a simple grep, I could only find a single initial data thorn that actually sets Avec, so the needed changes to public initial data thorns should be minimal. Further, I am under the impression that Avec evolutions are not yet supported in GRHydro anyway, and found only about 40 references to the Avec() gridfunctions within GRHydro, so renaming these variables to a new style would be trivial.
Please let me know your thoughts regarding this proposal. The IllinoisGRMHD code announcement paper is nearing completion, and at least having the infrastructure for this code in place by the next ET release would be great. In this regard, I have already asked Erik if he would be willing to merge the staggered prolongation/restriction code into the mainline Carpet for the next ET release. The soft freeze is Nov 10, these new features affect none of Carpet's other functionality, and I have thoroughly tested these new features with GRMHD evolutions, verifying that they work well.
Please consider the possibility of adding these small infrastructure enhancements prior to the freeze.
Sincerely,
-Zach
Zachariah Etienne Assistant Professor of Mathematics West Virginia University
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Hi Erik,
On Sun, Nov 2, 2014 at 11:24 AM, Erik Schnetter schnetter@cct.lsu.edu wrote:
Whether A^i lives on the same grid points as the metric or is staggered is part of its definition. Differing definitions should be stored in different grid functions, even if this seems to lead to a duplication. Cactus used to have different definitions of the metric, and used to store them in the same grid functions. There was a (slightly complex) mechanism that defined and recorded which definition was currently active. This led to errors, as people either didn't get the mechanism for choosing the metric definition right, or didn't add the respective checks and conversions into their code.
Defining a new grid function is cheap, and we should store the different discretizations of A^i in such a way.
Sounds good.
Currently, Carpet cannot handle different prolongation operators for different grid variables in the same group. This should probably change. (Does it with your proposed changes?) We also need to see how this affects symmetry conditions.
Currently in IllinoisGRMHD, we define four groups for A_{\mu}, each containing one gridfunction that stores a single component of A_{\mu}. We do this because each component is staggered differently, and Carpet does not support individually different staggerings for variables within a single group, even with our changes. As a first step, I prefer keeping it this way, so as to avoid introducing any bug into Carpet prior to release, but in the end it's up to you.
Regarding symmetry conditions, we had to write our own symmetry conditions to handle A_{\mu}, and at best only bitant symmetry across the z=0 plane is supported. Better symmetry support is planned for IllinoisGRMHD, moving away from the CartSymGN() function calls. In the meantime, IllinoisGRMHD is in a feature freeze until the code announcement paper is completed.
-Zach
On Thu, Oct 30, 2014 at 1:51 PM, Zach Etienne zachetie@gmail.com wrote:
Dear ET folks,
I have written a small thorn that converts HydroBase and ADMBase
variables
into the variables IllinoisGRMHD needs to perform a GRMHD evolution. This conversion works just fine on all variables... except the vector
potential.
As you may be aware, IllinoisGRMHD requires that A_t (=\sqrt{\gamma}
\Phi),
A_x, A_y, and A_z each be staggered differently (a la Table I of http://arxiv.org/abs/1007.2848).
However, HydroBase does not support defining such staggered A-fields.
Thus
it is currently impossible to modify/create GRMHD initial data thorns so that vector potential data produced by them can be directly read in by IllinoisGRMHD. For example, if an initial data thorn only outputs the
vector
potential on grid vertices, then an interpolation will need to be
performed
to staggered gridpoints. This will introduce truncation errors that, depending on the A-field structure and needed grid resolutions, may be severe. I conclude that this is _not_ The Right Strategy. The Right
Strategy
would not break any existing thorn as a first step, either.
So as a first step, I believe we should modify HydroBase so that
- A new set of four, 3-timelevel groups are defined, one for each
component
of the vector potential 2) The prolongation type can be specified differently for each of the
four
vector potential gridfunction groups, allowing for arbitrary staggerings
of
the single variable within each of these groups.
Variables in these groups will be read in by IllinoisGRMHD and can be
used
directly for evolutions.
Regarding the longer-term strategy, we could either
- modify Carpet so that different components of Avec() (the current
3-vector potential group in HydroBase) can accept different prolongation/restriction operators (then modifying IllinoisGRMHD to
accept
this new vector gridfunction), or
- just keep the four, 3-timelevel groups and phase out Avec() and Aphi
altogether.
Even the latter strategy would not be difficult to implement in current publicly available thorns:
After a simple grep, I could only find a single initial data thorn that actually sets Avec, so the needed changes to public initial data thorns should be minimal. Further, I am under the impression that Avec
evolutions
are not yet supported in GRHydro anyway, and found only about 40
references
to the Avec() gridfunctions within GRHydro, so renaming these variables
to a
new style would be trivial.
Please let me know your thoughts regarding this proposal. The
IllinoisGRMHD
code announcement paper is nearing completion, and at least having the infrastructure for this code in place by the next ET release would be
great.
In this regard, I have already asked Erik if he would be willing to merge the staggered prolongation/restriction code into the mainline Carpet for
the
next ET release. The soft freeze is Nov 10, these new features affect
none
of Carpet's other functionality, and I have thoroughly tested these new features with GRMHD evolutions, verifying that they work well.
Please consider the possibility of adding these small infrastructure enhancements prior to the freeze.
Sincerely,
-Zach
Zachariah Etienne Assistant Professor of Mathematics West Virginia University
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
-- Erik Schnetter schnetter@cct.lsu.edu http://www.perimeterinstitute.ca/personal/eschnetter/
users@lists.einsteintoolkit.org
-
Erik Schnetter -
Zach Etienne