hi all,
we've recently added the option to compute the Gauss constraint in the ProcaEvolve thorn (https://bitbucket.org/canuda/proca/src/experimental_miguel/ProcaEvolve/) and we were surprised to see that, even for very small evolutions of a simple (unperturbed) charged black hole, this constraint violation does not converge. of course everything is converging just fine at t=0, but once the evolution starts, convergence is lost really quickly.
after experimenting with different things, i think i've narrowed the issue down to the subcycling in time. i've attached some plots as well as the corresponding parameter files. these plots are not convergence tests, it's just what i'm observing with and without subcycling in time (everything else is the same as you can see from the parfiles).
these are evolutions for a single charged BH, with Q=0.2. in one of the parfiles (LeanBSSN_RN_Q0.2_nosubcycl_hf40_202509.par) all timelevels update at the same rate with
Carpet::time_refinement_factors = "[1, 1, 1, 1, 1, 1, 1]"
for the other parfile (LeanBSSN_RN_Q0.2_hf40_202509.par), all timelevels update at the same rate *except* the two inner ones. for the "base" grid functions, i see no noticeable difference insofar as i've checked, but for the Gauss constraint violation the results are very different as you can see from the plots attached:
- in the gc_x_RN_nosubcycl.pdf plot, everything looks fine - in the gc_x_RN_subcycl.pdf case, notice how a lot of noise propagates from the two inner refinements levels -- the ones that updated more frequently than the rest.
the problem is that this noise is not contained in the buffer region (if it was, we'd be fine), but propagates out, contaminates the rest of the grid, and totally ruins any convergence study.
actually a similar thing also happens for the hamiltonian constraint -- plots also attached (hc_x_RN_nosubcycl.pdf, hc_x_RN_subcycl.pdf) -- but in this case the effect is smaller and it doesn't spoil convergence studies.
so i was wondering whether this is at all expected and/or if there are some parameters or setting that we may have overlooked?
thanks, Miguel
Miguel
If I recall correctly, Ian Hinder studied convergence of black hole simulations with subcycling in time in great detail. The Einstein Toolkit gallery example for GW150914 contains the respective distilled knowledge. https://einsteintoolkit.org/gallery/bbh/index.html
Some important details that I recall: - You can regrid only when the fine and coarse grids are aligned - You cannot use time interpolation at all. You need to use enough buffer zones for all the RK substeps for all the fine timesteps for each coarse time step. With 3 ghost zones and RK4 you need 21 buffer zones.
You likely do not need to evolve for a long time to see loss of convergence. It might suffice to run for just a few time steps (a few coarse grid steps) and study convergence of one of the evolved variables (e.g. the lapse) in great detail over a wide range of resolutions. For this you do not need a large domain, nor do you need many refinement levels, nor do you need many grid points. For example, studying convergence near the point (1,0,0) requires probably about 100^3 points for each resolution in a two-level setup.
-erik
On Nov 19, 2025, at 06:17, Miguel Zilhão mzilhao@ua.pt wrote:
hi all,
we've recently added the option to compute the Gauss constraint in the ProcaEvolve thorn (https://bitbucket.org/canuda/proca/src/experimental_miguel/ProcaEvolve/) and we were surprised to see that, even for very small evolutions of a simple (unperturbed) charged black hole, this constraint violation does not converge. of course everything is converging just fine at t=0, but once the evolution starts, convergence is lost really quickly.
after experimenting with different things, i think i've narrowed the issue down to the subcycling in time. i've attached some plots as well as the corresponding parameter files. these plots are not convergence tests, it's just what i'm observing with and without subcycling in time (everything else is the same as you can see from the parfiles).
these are evolutions for a single charged BH, with Q=0.2. in one of the parfiles (LeanBSSN_RN_Q0.2_nosubcycl_hf40_202509.par) all timelevels update at the same rate with
Carpet::time_refinement_factors = "[1, 1, 1, 1, 1, 1, 1]"
for the other parfile (LeanBSSN_RN_Q0.2_hf40_202509.par), all timelevels update at the same rate *except* the two inner ones. for the "base" grid functions, i see no noticeable difference insofar as i've checked, but for the Gauss constraint violation the results are very different as you can see from the plots attached:
- in the gc_x_RN_nosubcycl.pdf plot, everything looks fine
- in the gc_x_RN_subcycl.pdf case, notice how a lot of noise propagates from the two inner refinements levels -- the ones that updated more frequently than the rest.
the problem is that this noise is not contained in the buffer region (if it was, we'd be fine), but propagates out, contaminates the rest of the grid, and totally ruins any convergence study.
actually a similar thing also happens for the hamiltonian constraint -- plots also attached (hc_x_RN_nosubcycl.pdf, hc_x_RN_subcycl.pdf) -- but in this case the effect is smaller and it doesn't spoil convergence studies.
so i was wondering whether this is at all expected and/or if there are some parameters or setting that we may have overlooked?
thanks, Miguel <hc_x_RN_nosubcycl.pdf><hc_x_RN_subcycl.pdf><gc_x_RN_subcycl.pdf><gc_x_RN_nosubcycl.pdf><LeanBSSN_RN_Q0.2_hf40_202509.par><LeanBSSN_RN_Q0.2_nosubcycl_hf40_202509.par>_______________________________________________ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
On 11/19/2025 8:24 AM, Erik Schnetter wrote:
Miguel
If I recall correctly, Ian Hinder studied convergence of black hole simulations with subcycling in time in great detail. The Einstein Toolkit gallery example for GW150914 contains the respective distilled knowledge. https://einsteintoolkit.org/gallery/bbh/index.html
Some important details that I recall:
- You can regrid only when the fine and coarse grids are aligned
- You cannot use time interpolation at all. You need to use enough
buffer zones for all the RK substeps for all the fine timesteps for each coarse time step. With 3 ghost zones and RK4 you need 21 buffer zones.
Does no time interpolation mean no dense output? That didn't exist when Ian did these tests, right?
--Steve
You likely do not need to evolve for a long time to see loss of convergence. It might suffice to run for just a few time steps (a few coarse grid steps) and study convergence of one of the evolved variables (e.g. the lapse) in great detail over a wide range of resolutions. For this you do not need a large domain, nor do you need many refinement levels, nor do you need many grid points. For example, studying convergence near the point (1,0,0) requires probably about 100^3 points for each resolution in a two-level setup.
-erik
On Nov 19, 2025, at 06:17, Miguel Zilhão mzilhao@ua.pt wrote:
hi all,
we've recently added the option to compute the Gauss constraint in the ProcaEvolve thorn (https://bitbucket.org/canuda/proca/src/experimental_miguel/ProcaEvolve/) and we were surprised to see that, even for very small evolutions of a simple (unperturbed) charged black hole, this constraint violation does not converge. of course everything is converging just fine at t=0, but once the evolution starts, convergence is lost really quickly.
after experimenting with different things, i think i've narrowed the issue down to the subcycling in time. i've attached some plots as well as the corresponding parameter files. these plots are not convergence tests, it's just what i'm observing with and without subcycling in time (everything else is the same as you can see from the parfiles).
these are evolutions for a single charged BH, with Q=0.2. in one of the parfiles (LeanBSSN_RN_Q0.2_nosubcycl_hf40_202509.par) all timelevels update at the same rate with
Carpet::time_refinement_factors = "[1, 1, 1, 1, 1, 1, 1]"
for the other parfile (LeanBSSN_RN_Q0.2_hf40_202509.par), all timelevels update at the same rate *except* the two inner ones. for the "base" grid functions, i see no noticeable difference insofar as i've checked, but for the Gauss constraint violation the results are very different as you can see from the plots attached:
- in the gc_x_RN_nosubcycl.pdf plot, everything looks fine
- in the gc_x_RN_subcycl.pdf case, notice how a lot of noise
propagates from the two inner refinements levels -- the ones that updated more frequently than the rest.
the problem is that this noise is not contained in the buffer region (if it was, we'd be fine), but propagates out, contaminates the rest of the grid, and totally ruins any convergence study.
actually a similar thing also happens for the hamiltonian constraint -- plots also attached (hc_x_RN_nosubcycl.pdf, hc_x_RN_subcycl.pdf) -- but in this case the effect is smaller and it doesn't spoil convergence studies.
so i was wondering whether this is at all expected and/or if there are some parameters or setting that we may have overlooked?
thanks, Miguel <hc_x_RN_nosubcycl.pdf><hc_x_RN_subcycl.pdf><gc_x_RN_subcycl.pdf><gc_x_RN_nosubcycl.pdf><LeanBSSN_RN_Q0.2_hf40_202509.par><LeanBSSN_RN_Q0.2_nosubcycl_hf40_202509.par>_______________________________________________ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
On Nov 19, 2025, at 10:26, Steven Brandt via Users users@einsteintoolkit.org wrote:
On 11/19/2025 8:24 AM, Erik Schnetter wrote:
Miguel
If I recall correctly, Ian Hinder studied convergence of black hole simulations with subcycling in time in great detail. The Einstein Toolkit gallery example for GW150914 contains the respective distilled knowledge. https://einsteintoolkit.org/gallery/bbh/index.html
Some important details that I recall:
- You can regrid only when the fine and coarse grids are aligned
- You cannot use time interpolation at all. You need to use enough buffer zones for all the RK substeps for all the fine timesteps for each coarse time step. With 3 ghost zones and RK4 you need 21 buffer zones.
Does no time interpolation mean no dense output? That didn't exist when Ian did these tests, right?
Output doesn't affect time evolution, so it doesn't matter which way you output things. Of course, if you use second-order accurate interpolation to output a quantity you cannot expect 4th order convergence for these quantities. If you output time-interpolated values of e.g. the lapse then you should check convergence only for the fine grid values of the lapse there, not for the interpolated coarse grid values.
-erik
hi Erik, all,
thanks for your input. for this specific example the grid is not moving at all, so i was expecting it to be a simpler setting than the case with binary BHs...
is it then expected to see all that noise propagating out of the buffer region and contaminating the whole grid in such a short amount of time (plot attached)? this is a bit uncomfortable, since the convergence on the electric constraint violation is completely lost... if it were a matter of dropping from 4th to 2nd order (as is the case in the L2 norm of the Hamiltonian constraint) i'd be fine with it, but a complete lack of convergence is difficult to justify...
thanks, Miguel
On 19/11/2025 16:23, Erik Schnetter wrote:
On Nov 19, 2025, at 10:26, Steven Brandt via Users users@einsteintoolkit.org wrote:
On 11/19/2025 8:24 AM, Erik Schnetter wrote:
Miguel
If I recall correctly, Ian Hinder studied convergence of black hole simulations with subcycling in time in great detail. The Einstein Toolkit gallery example for GW150914 contains the respective distilled knowledge. https://einsteintoolkit.org/gallery/bbh/index.html
Some important details that I recall:
- You can regrid only when the fine and coarse grids are aligned
- You cannot use time interpolation at all. You need to use enough
buffer zones for all the RK substeps for all the fine timesteps for each coarse time step. With 3 ghost zones and RK4 you need 21 buffer zones.
Does no time interpolation mean no dense output? That didn't exist when Ian did these tests, right?
Output doesn't affect time evolution, so it doesn't matter which way you output things. Of course, if you use second-order accurate interpolation to output a quantity you cannot expect 4th order convergence for these quantities. If you output time-interpolated values of e.g. the lapse then you should check convergence only for the fine grid values of the lapse there, not for the interpolated coarse grid values.
-erik
-- Erik Schnetter schnetter@gmail.com http://www.perimeterinstitute.ca/personal/eschnetter/
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
hi all,
i haven’t been able to join the weekly calls lately, but i wanted to bring this up again.
i recall that the problem is that, unless i’m missing something, the electric constraint isn’t converging at all in a very simple setting of a static charged black hole setup. this could be a real issue, especially since referees often ask for convergence tests and plots (and thus this could affect potential publications).
i’m happy to help dig into this further, but i’m not super familiar with Carpet's internals. should i file a bug report, or is someone already looking into it?
thanks, Miguel
On 19/11/2025 17:33, users-bounces@einsteintoolkit.org wrote:
hi Erik, all,
thanks for your input. for this specific example the grid is not moving at all, so i was expecting it to be a simpler setting than the case with binary BHs...
is it then expected to see all that noise propagating out of the buffer region and contaminating the whole grid in such a short amount of time (plot attached)? this is a bit uncomfortable, since the convergence on the electric constraint violation is completely lost... if it were a matter of dropping from 4th to 2nd order (as is the case in the L2 norm of the Hamiltonian constraint) i'd be fine with it, but a complete lack of convergence is difficult to justify...
thanks, Miguel
On 19/11/2025 16:23, Erik Schnetter wrote:
On Nov 19, 2025, at 10:26, Steven Brandt via Users users@einsteintoolkit.org wrote:
On 11/19/2025 8:24 AM, Erik Schnetter wrote:
Miguel
If I recall correctly, Ian Hinder studied convergence of black hole simulations with subcycling in time in great detail. The Einstein Toolkit gallery example for GW150914 contains the respective distilled knowledge. https://einsteintoolkit.org/gallery/bbh/index.html
Some important details that I recall:
- You can regrid only when the fine and coarse grids are aligned
- You cannot use time interpolation at all. You need to use enough
buffer zones for all the RK substeps for all the fine timesteps for each coarse time step. With 3 ghost zones and RK4 you need 21 buffer zones.
Does no time interpolation mean no dense output? That didn't exist when Ian did these tests, right?
Output doesn't affect time evolution, so it doesn't matter which way you output things. Of course, if you use second-order accurate interpolation to output a quantity you cannot expect 4th order convergence for these quantities. If you output time-interpolated values of e.g. the lapse then you should check convergence only for the fine grid values of the lapse there, not for the interpolated coarse grid values.
-erik
-- Erik Schnetter schnetter@gmail.com http://www.perimeterinstitute.ca/personal/eschnetter/
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Hello Miguel,
i’m happy to help dig into this further, but i’m not super familiar with Carpet's internals. should i file a bug report, or is someone already looking into it?
Sure.
Yours, Roland
i’m happy to help dig into this further, but i’m not super familiar with Carpet's internals. should i file a bug report, or is someone already looking into it?
Sure.
i've just submitted a bug report here: https://bitbucket.org/einsteintoolkit/tickets/issues/2920 i added all the information needed to understand and reproduce the problem, but please let me know if there's anything missing.
thanks, Miguel
users@lists.einsteintoolkit.org
-
Erik Schnetter -
Miguel Zilhão -
Roland Haas -
Steven Brandt