Grid function "AHFINDERDIRECT::ahmask" has only 1 active time levels on refinement level 1; this is not enough for time interpolation
Hi,
I am getting the following error in a BBH simulation just before the merger:
WARNING level 0 in thorn CarpetInterp processor 33 host c438-302.stampede.tacc.utexas.edu (line 1698 of /work/00915/hinder/Cactus/AEI/arrangements/Carpet/CarpetInterp/src/interp.cc): -> Grid function "AHFINDERDIRECT::ahmask" has only 1 active time levels on refinement level 1; this is not enough for time interpolation cactus_sim: /work/00915/hinder/Cactus/AEI/arrangements/Carpet/Carpet/src/helpers.cc:314: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.
Does anyone know why this happens, and if there is a workaround I can use to get the run going again? Different resolutions of this run, as well as other runs with similar parameter files, have worked fine. It looks like an edge case. I am using the trunk ET from 16-Apr-2013.
Relevant tickets:
https://trac.einsteintoolkit.org/ticket/237 (closed) https://trac.einsteintoolkit.org/ticket/380 (still open)
I have ADMBase::metric_timelevels set to the default, which is 1. Should it be 3? Would it work to recover with this parameter set to 3? The parameter claims to be steerable, though I don't know what it would use for the previously nonexistent timelevels.
How about changing the frequency of AHFinderDirect output to match the coarsest timelevel?
On Tue, May 14, 2013 at 08:35:23AM +0200, Ian Hinder wrote:
(line 1698 of /work/00915/hinder/Cactus/AEI/arrangements/Carpet/CarpetInterp/src/interp.cc): -> Grid function "AHFINDERDIRECT::ahmask" has only 1 active time levels on refinement level 1; this is not enough for time interpolation
Does anyone know why this happens
Something tries to interpolate the mask. Evolution shouldn't do that, the mask is supposed to be written each (sub-)timestep AFAIR. Now, there are other possibilities, as you mentioned below...
and if there is a workaround I can use to get the run going again?
The questions is what makes Carpet interpolate, and avoid that.
I have ADMBase::metric_timelevels set to the default, which is 1. Should it be 3?
It should work with 1.
How about changing the frequency of AHFinderDirect output to match the coarsest timelevel?
I suspect that this is the cause of your problem: you output in between evolution times of the coarsest level, making Carpet interpolate the mask. You could (only for the mask probably), output only at full coarse level steps, if don't need it more often. You could also try (although here I am less sure that this will prevent interpolation) to restrict the output of the mask to some of the finer levels only, at full time steps of these.
Frank
On 14 May 2013, at 15:20, Frank Loeffler knarf@cct.lsu.edu wrote:
On Tue, May 14, 2013 at 08:35:23AM +0200, Ian Hinder wrote:
(line 1698 of /work/00915/hinder/Cactus/AEI/arrangements/Carpet/CarpetInterp/src/interp.cc): -> Grid function "AHFINDERDIRECT::ahmask" has only 1 active time levels on refinement level 1; this is not enough for time interpolation
Does anyone know why this happens
Something tries to interpolate the mask. Evolution shouldn't do that, the mask is supposed to be written each (sub-)timestep AFAIR. Now, there are other possibilities, as you mentioned below...
and if there is a workaround I can use to get the run going again?
The questions is what makes Carpet interpolate, and avoid that.
I have ADMBase::metric_timelevels set to the default, which is 1. Should it be 3?
It should work with 1.
How about changing the frequency of AHFinderDirect output to match the coarsest timelevel?
I suspect that this is the cause of your problem: you output in between evolution times of the coarsest level, making Carpet interpolate the mask. You could (only for the mask probably), output only at full coarse level steps, if don't need it more often. You could also try (although here I am less sure that this will prevent interpolation) to restrict the output of the mask to some of the finer levels only, at full time steps of these.
I'm not interested in the mask, and I don't output it. Would something still be trying to interpolate this mask?
This mask is intended for excision. With excision, AHFinderDirect needs to make sure that it does not interpolate from within the excised region. AHFinderDirect copies the excision mask into ahmask, and then interpolates from ahmask onto the horizon surface. It can thus find out whether an excised point was used for interpolation. Unfortunately, AHFinderDirect's code is a bit inflexible, and introducing a mask that is always there is much easier than changing the number of variables that are interpolated.
This interpolation is the same as is used for the ADMBase variables, and ahmask thus should have the same number of time levels. You are probably setting the number of ADMBase timelevels to 1, but some thorn (probably McLachlan) needs more timelevels and thus increases this number automatically.
If you set the number of ADMBase timelevels explicitly to the number you need (probably 3), then ahmask will also have more timelevels, and things should work. Since you are presumably already using 3 ADMBase timelevels, this will not change your evolution. And since ahmask will be all zeros anyway, increasing its number of timelevels should have no ill effect.
If you are looking for horizons at times that require time interpolation, then 1 timelevel for the ADMBase variables does not suffice.
-erik
On Tue, May 14, 2013 at 9:26 AM, Ian Hinder ian.hinder@aei.mpg.de wrote:
On 14 May 2013, at 15:20, Frank Loeffler knarf@cct.lsu.edu wrote:
On Tue, May 14, 2013 at 08:35:23AM +0200, Ian Hinder wrote:
(line 1698 of /work/00915/hinder/Cactus/AEI/arrangements/Carpet/CarpetInterp/src/interp.cc):
-> Grid function "AHFINDERDIRECT::ahmask" has only 1 active time levels on refinement level 1; this is not enough for time interpolation
Does anyone know why this happens
Something tries to interpolate the mask. Evolution shouldn't do that, the mask is supposed to be written each (sub-)timestep AFAIR. Now, there are other possibilities, as you mentioned below...
and if there is a workaround I can use to get the run going again?
The questions is what makes Carpet interpolate, and avoid that.
I have ADMBase::metric_timelevels set to the default, which is 1. Should it be 3?
It should work with 1.
How about changing the frequency of AHFinderDirect output to match the coarsest timelevel?
I suspect that this is the cause of your problem: you output in between evolution times of the coarsest level, making Carpet interpolate the mask. You could (only for the mask probably), output only at full coarse level steps, if don't need it more often. You could also try (although here I am less sure that this will prevent interpolation) to restrict the output of the mask to some of the finer levels only, at full time steps of these.
I'm not interested in the mask, and I don't output it. Would something still be trying to interpolate this mask?
-- Ian Hinder http://numrel.aei.mpg.de/people/hinder
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
On 14 May 2013, at 16:51, Erik Schnetter schnetter@cct.lsu.edu wrote:
This mask is intended for excision. With excision, AHFinderDirect needs to make sure that it does not interpolate from within the excised region. AHFinderDirect copies the excision mask into ahmask, and then interpolates from ahmask onto the horizon surface. It can thus find out whether an excised point was used for interpolation. Unfortunately, AHFinderDirect's code is a bit inflexible, and introducing a mask that is always there is much easier than changing the number of variables that are interpolated.
This interpolation is the same as is used for the ADMBase variables, and ahmask thus should have the same number of time levels. You are probably setting the number of ADMBase timelevels to 1, but some thorn (probably McLachlan) needs more timelevels and thus increases this number automatically.
In my case, I didn't think about it, and left the parameter set to the default. McLachlan then asks for storage for more timelevels.
If you set the number of ADMBase timelevels explicitly to the number you need (probably 3), then ahmask will also have more timelevels, and things should work. Since you are presumably already using 3 ADMBase timelevels, this will not change your evolution. And since ahmask will be all zeros anyway, increasing its number of timelevels should have no ill effect.
Yes, this worked. Setting the parameter on recovery allowed the run to proceed.
Maybe the number of timelevels should be set to the number actually active for the metric, rather than the value of a parameter which might be overridden by something else?
If you are looking for horizons at times that require time interpolation, then 1 timelevel for the ADMBase variables does not suffice.
I hope that my horizons are always on the finest grid, so I shouldn't need to interpolate on the horizons themselves, but the horizon-finding algorithm might need to look further out.
-erik
On Tue, May 14, 2013 at 9:26 AM, Ian Hinder ian.hinder@aei.mpg.de wrote:
On 14 May 2013, at 15:20, Frank Loeffler knarf@cct.lsu.edu wrote:
On Tue, May 14, 2013 at 08:35:23AM +0200, Ian Hinder wrote:
(line 1698 of /work/00915/hinder/Cactus/AEI/arrangements/Carpet/CarpetInterp/src/interp.cc): -> Grid function "AHFINDERDIRECT::ahmask" has only 1 active time levels on refinement level 1; this is not enough for time interpolation
Does anyone know why this happens
Something tries to interpolate the mask. Evolution shouldn't do that, the mask is supposed to be written each (sub-)timestep AFAIR. Now, there are other possibilities, as you mentioned below...
and if there is a workaround I can use to get the run going again?
The questions is what makes Carpet interpolate, and avoid that.
I have ADMBase::metric_timelevels set to the default, which is 1. Should it be 3?
It should work with 1.
How about changing the frequency of AHFinderDirect output to match the coarsest timelevel?
I suspect that this is the cause of your problem: you output in between evolution times of the coarsest level, making Carpet interpolate the mask. You could (only for the mask probably), output only at full coarse level steps, if don't need it more often. You could also try (although here I am less sure that this will prevent interpolation) to restrict the output of the mask to some of the finer levels only, at full time steps of these.
I'm not interested in the mask, and I don't output it. Would something still be trying to interpolate this mask?
-- Ian Hinder http://numrel.aei.mpg.de/people/hinder
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
-- Erik Schnetter schnetter@cct.lsu.edu http://www.perimeterinstitute.ca/personal/eschnetter/
users@lists.einsteintoolkit.org
-
Erik Schnetter -
Frank Loeffler -
Ian Hinder