Hi,
I was looking for advice on implementing an integrator for timelike and null geodesics during an ET simulation. The confusion I have is how to deal with AMR. Ideally, I'd like to use RK4 and MoL, but I foresee several issues with geodesics crossing from coarser to finer and finer to coarser zones. I do have a horrible hack that may possibly be used, but it seems overly convoluted to me. Basically, I update the grid arrays in MoL in local mode, and unless the local grid happens to be the finest grid that owns the geodesic, I set RHS to zero (there are additional hacks because MoL copies the past time level onto the current one automatically). But still there is the issue of geodesics crossing AMR boundaries. In particular, when moving from fine to coarse, the geodesic may end up being behind because the finer grid may be at an earlier time than the coarser one.
One possible workaround would be to update the geodesics during every iteration on the finest level. This would imply that I would need to interpolate the metric, shift, and lapse (and their derivatives) during each ministep of the finest level even if the point being interpolated doesn't exist on the finest level. This, in turn, means that the carpet would need to interpolate those points in time during the ministep where the local time will at some intermediate time between cctk_time(old) and cctk_time(new). I don't think this works correctly in the current version of carpet.
An alternative method I was considering was a second-order in time integration. For this, I would need to be able to interpolate the metric and its derivatives during the current (finest level) time and the previous (finest level) time. For the moment, I'd be happy to assume that the finest refinement level will not change during the evolution.
On 6 Aug 2015, at 16:48, Yosef Zlochower yosef@astro.rit.edu wrote:
Hi,
I was looking for advice on implementing an integrator for timelike and null geodesics during an ET simulation. The confusion I have is how to deal with AMR. Ideally, I'd like to use RK4 and MoL, but I foresee several issues with geodesics crossing from coarser to finer and finer to coarser zones. I do have a horrible hack that may possibly be used, but it seems overly convoluted to me. Basically, I update the grid arrays in MoL in local mode, and unless the local grid happens to be the finest grid that owns the geodesic, I set RHS to zero (there are additional hacks because MoL copies the past time level onto the current one automatically). But still there is the issue of geodesics crossing AMR boundaries. In particular, when moving from fine to coarse, the geodesic may end up being behind because the finer grid may be at an earlier time than the coarser one.
One possible workaround would be to update the geodesics during every iteration on the finest level. This would imply that I would need to interpolate the metric, shift, and lapse (and their derivatives) during each ministep of the finest level even if the point being interpolated doesn't exist on the finest level.
This, in turn, means that the carpet would need to interpolate those points in time during the ministep where the local time will at some intermediate time between cctk_time(old) and cctk_time(new). I don't think this works correctly in the current version of carpet.
Correct. The time used by the interpolator for time interpolation is not consistently cctk_time, so when MoL modified this variable during MoL_CalcRHS, this is ignored by CarpetInterp. See the discussion in https://trac.einsteintoolkit.org/ticket/1656, and a patch with a parameter which makes it use cctk_time.
An alternative method I was considering was a second-order in time integration. For this, I would need to be able to interpolate the metric and its derivatives during the current (finest level) time and the previous (finest level) time. For the moment, I'd be happy to assume that the finest refinement level will not change during the evolution.
"It's complicated".
I describe an approach in http://lists.einsteintoolkit.org/pipermail/users/2013-November/003280.html (the archive page seems to not know about line wrapping).
Thanks for the help. I guess if I used interpolation in time the geodesics would only be accurate to 2nd order.
On 08/11/2015 09:15 AM, Ian Hinder wrote:
On 6 Aug 2015, at 16:48, Yosef Zlochower <yosef@astro.rit.edu mailto:yosef@astro.rit.edu> wrote:
Hi,
I was looking for advice on implementing an integrator for timelike and null geodesics during an ET simulation. The confusion I have is how to deal with AMR. Ideally, I'd like to use RK4 and MoL, but I foresee several issues with geodesics crossing from coarser to finer and finer to coarser zones. I do have a horrible hack that may possibly be used, but it seems overly convoluted to me. Basically, I update the grid arrays in MoL in local mode, and unless the local grid happens to be the finest grid that owns the geodesic, I set RHS to zero (there are additional hacks because MoL copies the past time level onto the current one automatically). But still there is the issue of geodesics crossing AMR boundaries. In particular, when moving from fine to coarse, the geodesic may end up being behind because the finer grid may be at an earlier time than the coarser one.
One possible workaround would be to update the geodesics during every iteration on the finest level. This would imply that I would need to interpolate the metric, shift, and lapse (and their derivatives) during each ministep of the finest level even if the point being interpolated doesn't exist on the finest level.
This, in turn, means that the carpet would need to interpolate those points in time during the ministep where the local time will at some intermediate time between cctk_time(old) and cctk_time(new). I don't think this works correctly in the current version of carpet.
Correct. The time used by the interpolator for time interpolation is not consistently cctk_time, so when MoL modified this variable during MoL_CalcRHS, this is ignored by CarpetInterp. See the discussion in https://trac.einsteintoolkit.org/ticket/1656, and a patch with a parameter which makes it use cctk_time.
An alternative method I was considering was a second-order in time integration. For this, I would need to be able to interpolate the metric and its derivatives during the current (finest level) time and the previous (finest level) time. For the moment, I'd be happy to assume that the finest refinement level will not change during the evolution.
"It's complicated".
I describe an approach in http://lists.einsteintoolkit.org/pipermail/users/2013-November/003280.html (the archive page seems to not know about line wrapping).
-- Ian Hinder http://members.aei.mpg.de/ianhin
I'm working on a thorn that uses local interpolations to get the RHS for the geodesic evolution. I have an issue when points get too close (or inside of) buffer zones (because then the RHS won't contain correct data during the MoL ministeps). How can I determine if the interpolation stencil around a given point contains any buffer zone points?
On 08/11/2015 03:06 PM, Yosef Zlochower wrote:
Thanks for the help. I guess if I used interpolation in time the geodesics would only be accurate to 2nd order.
On 08/11/2015 09:15 AM, Ian Hinder wrote:
On 6 Aug 2015, at 16:48, Yosef Zlochower <yosef@astro.rit.edu mailto:yosef@astro.rit.edu> wrote:
Hi,
I was looking for advice on implementing an integrator for timelike and null geodesics during an ET simulation. The confusion I have is how to deal with AMR. Ideally, I'd like to use RK4 and MoL, but I foresee several issues with geodesics crossing from coarser to finer and finer to coarser zones. I do have a horrible hack that may possibly be used, but it seems overly convoluted to me. Basically, I update the grid arrays in MoL in local mode, and unless the local grid happens to be the finest grid that owns the geodesic, I set RHS to zero (there are additional hacks because MoL copies the past time level onto the current one automatically). But still there is the issue of geodesics crossing AMR boundaries. In particular, when moving from fine to coarse, the geodesic may end up being behind because the finer grid may be at an earlier time than the coarser one.
One possible workaround would be to update the geodesics during every iteration on the finest level. This would imply that I would need to interpolate the metric, shift, and lapse (and their derivatives) during each ministep of the finest level even if the point being interpolated doesn't exist on the finest level.
This, in turn, means that the carpet would need to interpolate those points in time during the ministep where the local time will at some intermediate time between cctk_time(old) and cctk_time(new). I don't think this works correctly in the current version of carpet.
Correct. The time used by the interpolator for time interpolation is not consistently cctk_time, so when MoL modified this variable during MoL_CalcRHS, this is ignored by CarpetInterp. See the discussion in https://trac.einsteintoolkit.org/ticket/1656, and a patch with a parameter which makes it use cctk_time.
An alternative method I was considering was a second-order in time integration. For this, I would need to be able to interpolate the metric and its derivatives during the current (finest level) time and the previous (finest level) time. For the moment, I'd be happy to assume that the finest refinement level will not change during the evolution.
"It's complicated".
I describe an approach in http://lists.einsteintoolkit.org/pipermail/users/2013-November/003280.html (the archive page seems to not know about line wrapping).
-- Ian Hinder http://members.aei.mpg.de/ianhin
Yosef
One easy way is to look at CarpetReduce: It will ignore points that are invalid. There is a grid function CarpetReduce::weight that specifies how much a point should contribute towards reductions; if the weight is zero, then the points is either an outer boundary, a buffer zone, or will be overwritten by restriction. The sum of all weights (modulo the level's grid spacing) is the volume of the simulation domain. The weight is 1 for "regular points", 0 for points that should be ignored, and a fraction of 1 for points near refinement and near outer boundaries.
I don't recall whether the weight is zero in inter-process boundaries and symmetry boundaries; it may not be, but you can use cctk_bbox and cctk_nghostzones in the usual way to exclude these.
-erik
On Wed, Aug 19, 2015 at 10:51 AM, Yosef Zlochower yosef@astro.rit.edu wrote:
I'm working on a thorn that uses local interpolations to get the RHS for the geodesic evolution. I have an issue when points get too close (or inside of) buffer zones (because then the RHS won't contain correct data during the MoL ministeps). How can I determine if the interpolation stencil around a given point contains any buffer zone points?
On 08/11/2015 03:06 PM, Yosef Zlochower wrote:
Thanks for the help. I guess if I used interpolation in time the geodesics would only be accurate to 2nd order.
On 08/11/2015 09:15 AM, Ian Hinder wrote:
On 6 Aug 2015, at 16:48, Yosef Zlochower <yosef@astro.rit.edu mailto:yosef@astro.rit.edu> wrote:
Hi,
I was looking for advice on implementing an integrator for timelike and null geodesics during an ET simulation. The confusion I have is how to deal with AMR. Ideally, I'd like to use RK4 and MoL, but I foresee several issues with geodesics crossing from coarser to finer and finer to coarser zones. I do have a horrible hack that may possibly be used, but it seems overly convoluted to me. Basically, I update the grid arrays in MoL in local mode, and unless the local grid happens to be the finest grid that owns the geodesic, I set RHS to zero (there are additional hacks because MoL copies the past time level onto the current one automatically). But still there is the issue of geodesics crossing AMR boundaries. In particular, when moving from fine to coarse, the geodesic may end up being behind because the finer grid may be at an earlier time than the coarser one.
One possible workaround would be to update the geodesics during every iteration on the finest level. This would imply that I would need to interpolate the metric, shift, and lapse (and their derivatives) during each ministep of the finest level even if the point being interpolated doesn't exist on the finest level.
This, in turn, means that the carpet would need to interpolate those points in time during the ministep where the local time will at some intermediate time between cctk_time(old) and cctk_time(new). I don't think this works correctly in the current version of carpet.
Correct. The time used by the interpolator for time interpolation is not consistently cctk_time, so when MoL modified this variable during MoL_CalcRHS, this is ignored by CarpetInterp. See the discussion in https://trac.einsteintoolkit.org/ticket/1656, and a patch with a parameter which makes it use cctk_time.
An alternative method I was considering was a second-order in time integration. For this, I would need to be able to interpolate the metric and its derivatives during the current (finest level) time and the previous (finest level) time. For the moment, I'd be happy to assume that the finest refinement level will not change during the evolution.
"It's complicated".
I describe an approach in
http://lists.einsteintoolkit.org/pipermail/users/2013-November/003280.html
(the archive page seems to not know about line wrapping).
-- Ian Hinder http://members.aei.mpg.de/ianhin
-- Dr. Yosef Zlochower Center for Computational Relativity and Gravitation Associate Professor School of Mathematical Sciences Rochester Institute of Technology 85 Lomb Memorial Drive Rochester, NY 14623
Office:74-2067 Phone: +1 585-475-6103
yosef@astro.rit.edu
CONFIDENTIALITY NOTE: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and destroy any copies of this information. _______________________________________________ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Thanks Erik. This should work quite well for most cases. The only possible issue I see is when one is using a weight that is zero outside some given sphere.
On 08/19/2015 11:22 AM, Erik Schnetter wrote:
Yosef
One easy way is to look at CarpetReduce: It will ignore points that are invalid. There is a grid function CarpetReduce::weight that specifies how much a point should contribute towards reductions; if the weight is zero, then the points is either an outer boundary, a buffer zone, or will be overwritten by restriction. The sum of all weights (modulo the level's grid spacing) is the volume of the simulation domain. The weight is 1 for "regular points", 0 for points that should be ignored, and a fraction of 1 for points near refinement and near outer boundaries.
I don't recall whether the weight is zero in inter-process boundaries and symmetry boundaries; it may not be, but you can use cctk_bbox and cctk_nghostzones in the usual way to exclude these.
-erik
On Wed, Aug 19, 2015 at 10:51 AM, Yosef Zlochower <yosef@astro.rit.edu mailto:yosef@astro.rit.edu> wrote:
I'm working on a thorn that uses local interpolations to get the RHS for the geodesic evolution. I have an issue when points get too close (or inside of) buffer zones (because then the RHS won't contain correct data during the MoL ministeps). How can I determine if the interpolation stencil around a given point contains any buffer zone points? On 08/11/2015 03:06 PM, Yosef Zlochower wrote: > Thanks for the help. I guess if I used interpolation in time the > geodesics would only be accurate to 2nd order. > > On 08/11/2015 09:15 AM, Ian Hinder wrote: >> >> On 6 Aug 2015, at 16:48, Yosef Zlochower <yosef@astro.rit.edu <mailto:yosef@astro.rit.edu> >> <mailto:yosef@astro.rit.edu <mailto:yosef@astro.rit.edu>>> wrote: >> >>> Hi, >>> >>> I was looking for advice on implementing an integrator for timelike >>> and null geodesics during an ET simulation. The confusion I have is >>> how to deal with AMR. Ideally, I'd like to use RK4 and MoL, but I >>> foresee several issues with geodesics crossing from coarser to finer >>> and finer to coarser zones. I do have a horrible hack that may possibly >>> be used, but it seems overly convoluted to me. Basically, I update the >>> grid arrays in MoL in local mode, and unless the local grid happens to >>> be the finest grid that owns the geodesic, I set RHS to zero (there are >>> additional hacks because MoL copies the past time level onto the >>> current one automatically). But >>> still there is the issue of geodesics crossing AMR boundaries. In >>> particular, when moving from fine to coarse, the geodesic may end up >>> being behind because the finer grid may be at an earlier time than the >>> coarser one. >>> >>> One possible workaround would be to update >>> the geodesics during every iteration on the finest level. This would >>> imply that I would need to interpolate the metric, shift, and lapse >>> (and their derivatives) during each ministep of the finest level even >>> if the point being interpolated doesn't exist on the finest level. >> >>> This, in turn, means that the carpet would need to interpolate those >>> points in time during the ministep where the local time will at some >>> intermediate time between cctk_time(old) and cctk_time(new). I don't >>> think this works correctly in the current version of carpet. >> >> Correct. The time used by the interpolator for time interpolation is >> not consistently cctk_time, so when MoL modified this variable during >> MoL_CalcRHS, this is ignored by CarpetInterp. See the discussion in >> https://trac.einsteintoolkit.org/ticket/1656, and a patch with a >> parameter which makes it use cctk_time. >> >>> An alternative method I was considering was a second-order in time >>> integration. For this, I would need to be able to interpolate the >>> metric and its derivatives during the current (finest level) time and >>> the previous (finest level) time. For the moment, I'd be happy to >>> assume that the finest refinement level will not change during the >>> evolution. >> >> "It's complicated". >> >> I describe an approach in >> http://lists.einsteintoolkit.org/pipermail/users/2013-November/003280.html >> (the archive page seems to not know about line wrapping). >> >> -- >> Ian Hinder >> http://members.aei.mpg.de/ianhin >> > > -- Dr. Yosef Zlochower Center for Computational Relativity and Gravitation Associate Professor School of Mathematical Sciences Rochester Institute of Technology 85 Lomb Memorial Drive Rochester, NY 14623 Office:74-2067 Phone: +1 585-475-6103 <tel:%2B1%20585-475-6103> yosef@astro.rit.edu <mailto:yosef@astro.rit.edu> CONFIDENTIALITY NOTE: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and destroy any copies of this information. _______________________________________________ Users mailing list Users@einsteintoolkit.org <mailto:Users@einsteintoolkit.org> http://lists.einsteintoolkit.org/mailman/listinfo/users-- Erik Schnetter <schnetter@cct.lsu.edu mailto:schnetter@cct.lsu.edu> http://www.perimeterinstitute.ca/personal/eschnetter/
users@lists.einsteintoolkit.org
-
Erik Schnetter -
Ian Hinder -
Yosef Zlochower