Hello,
I'd like to propose the following patch for the schedule.ccl of thorn Exact:
------------------------------------------------------ 135a136,140
schedule Exact__initialize in MoL_PostStep { LANG: Fortran } "Set data from exact solution on an exact slice"
------------------------------------------------------
This should ensure that Exact__initialize is executed after regridding, in the same way that Exact__gauge already is.
Thanks! Eloisa
On 1 May 2010, at 00:20, Eloisa Bentivegna wrote:
Hello,
I'd like to propose the following patch for the schedule.ccl of thorn Exact:
135a136,140
schedule Exact__initialize in MoL_PostStep { LANG: Fortran } "Set data from exact solution on an exact slice"
This should ensure that Exact__initialize is executed after regridding, in the same way that Exact__gauge already is.
Would this function be a good candidate for the MoL_PseudoEvolution group? That group is intended for routines which "fake" an evolution, or more generally set grid functions which require the ghost and refinement boundary points to be filled correctly on regridding etc.
Hello all,
Would this function be a good candidate for the MoL_PseudoEvolution group? That group is intended for routines which "fake" an evolution, or more generally set grid functions which require the ghost and refinement boundary points to be filled correctly on regridding etc.
I have used Exact in a couple of the testsuites and I think moving Exact to Pseudoevolution (I think it fits the description) would require changes to all of these parameter files. Currently anything that runs in PostStep or PseudoEvolution itself runs after Exact, which is what we want, since Exact should behave just like an evolution thorn. While scheduling in PreStep is not perfectly equivalent to a real evolution, it would seem to cause fewer problems since only routines in MoL_PreStep and the ones inside of MoL's substep loop see anything different (namely the updated variables). If Exact itself moves to PseudoEvolution then one needs extra schedue statements to enforce the proper ordering.
I would also like to propose to schedule PseudoEvolution in PostRestrict after mol poststep so that it picks up any changes in variables that come from restriction (only important in the outer boundaries I think). Eg. a thorn copying say gxx into a second grid function GXX would either have to schedule boundary routines in Postrestrict (even though the copy operation itself is purely algebraic and pointwise) or would need to run after gxx has been restricted and the newly restricted values used to populate the outer and symmetry boundaries. WeylScal4 (which computes derivatives so needs boundary conditions anyway) had to do something like this.
Yours, Roland
Yours, Roland
Roland Haas ha scritto:
Hello all,
Would this function be a good candidate for the MoL_PseudoEvolution group? That group is intended for routines which "fake" an evolution, or more generally set grid functions which require the ghost and refinement boundary points to be filled correctly on regridding etc.
I have used Exact in a couple of the testsuites and I think moving Exact to Pseudoevolution (I think it fits the description) would require changes to all of these parameter files. Currently anything that runs in PostStep or PseudoEvolution itself runs after Exact, which is what we want, since Exact should behave just like an evolution thorn. While scheduling in PreStep is not perfectly equivalent to a real evolution, it would seem to cause fewer problems since only routines in MoL_PreStep and the ones inside of MoL's substep loop see anything different (namely the updated variables). If Exact itself moves to PseudoEvolution then one needs extra schedue statements to enforce the proper ordering.
Ian, Roland,
thanks for all the suggestions. I'm looking at the use of PseudoEvolution in my source tree and I see that some "analysis" functions are also scheduled in it (I looking, for instance, at ML_ADMConstraints). That, I guess, is one of the cases where scheduling Exact__initialize in this bin, without adding further constraints in the other functions scheduled here, would lead to incorrect behavior (Roland, do you have other examples?).
I would have thought, however, that any function that assumes that the evolution step has completed (such as one calculating the constraints) should be held until after pseudoevolution has completed. So PseudoEvolution does seem like the right place to schedule Exact__initialize, and if it breaks the tests it's only because of functions which shouldn't belong to this bin anyway. Any comments?
I would also like to propose to schedule PseudoEvolution in PostRestrict after mol poststep so that it picks up any changes in variables that come from restriction (only important in the outer boundaries I think). Eg. a thorn copying say gxx into a second grid function GXX would either have to schedule boundary routines in Postrestrict (even though the copy operation itself is purely algebraic and pointwise) or would need to run after gxx has been restricted and the newly restricted values used to populate the outer and symmetry boundaries. WeylScal4 (which computes derivatives so needs boundary conditions anyway) had to do something like this.
I'm not sure I understand this point. In your example, what's wrong with having the copy happen at postrestrict, like you say?
Eloisa
Hello Eloisa,
thanks for all the suggestions. I'm looking at the use of PseudoEvolution in my source tree and I see that some "analysis" functions are also scheduled in it (I looking, for instance, at ML_ADMConstraints). That, I guess, is one of the cases where scheduling Exact__initialize in this bin, without adding further constraints in the other functions scheduled here, would lead to incorrect behavior (Roland, do you have other examples?).
Not really. WeylScal4 is the only I use that schedules in PseudoEvolution. If a routine does only purely pointwise algebraic operations (a lot of our Hydro analysis thorns do) then scheduling in ANALYSIS would be best I think (happens after restriction, regridding etc are done, so only one call is needed). WeylScal is in PseudoEvolution since it takes derivatives and therefor has to be in Evol for prolongation to work properly (b/c of the order in which coarse and fine level routines end up being called in EVOL and ANALYSIS for the same cctk_time).
I would have thought, however, that any function that assumes that the evolution step has completed (such as one calculating the constraints) should be held until after pseudoevolution has completed. So PseudoEvolution does seem like the right place to schedule Exact__initialize, and if it breaks the tests it's only because of functions which shouldn't belong to this bin anyway. Any comments?
Anything that does something like BSSNtoADM [if it did not happen in MoL_Poststep] uses the evolved BSSN variables but is still part of the evolution I would think since for routines outside of the BSSN evolution the ADM variables look like the "evolving" variables.
I would also like to propose to schedule PseudoEvolution in PostRestrict after mol poststep so that it picks up any changes in variables that come from restriction (only important in the outer boundaries I think). Eg. a thorn copying say gxx into a second grid function GXX would either have to schedule boundary routines in Postrestrict (even though the copy operation itself is purely algebraic and pointwise) or would need to run after gxx has been restricted and the newly restricted values used to populate the outer and symmetry boundaries. WeylScal4 (which computes derivatives so needs boundary conditions anyway) had to do something like this.
I'm not sure I understand this point. In your example, what's wrong with having the copy happen at postrestrict, like you say?
Sorry, I was not clear enough it seems. My point is that routines that are algebraic and pointwise normally (should) do not need boundary/symmetry conditions since they should inherit them from their sources (assuming the routine runs after boundary conditions have been applied to the sources). Eg. BSSNtoADM does not need SYNC's if it runs after the boundaries have been applied to the BSSN variables [please correct me if I have this wrong]. Similarly for a simple Con2Prim routine for a hydro thorn.
Copying in postrestrict is fine (I think). Currently it would not happen since PseudoEvolution is not scheduled in the postrestrict bins.
Yours, Roland
On May 4, 2010, at 15:10 , Eloisa Bentivegna wrote:
Roland Haas ha scritto:
Hello all,
Would this function be a good candidate for the MoL_PseudoEvolution group? That group is intended for routines which "fake" an evolution, or more generally set grid functions which require the ghost and refinement boundary points to be filled correctly on regridding etc.
I have used Exact in a couple of the testsuites and I think moving Exact to Pseudoevolution (I think it fits the description) would require changes to all of these parameter files. Currently anything that runs in PostStep or PseudoEvolution itself runs after Exact, which is what we want, since Exact should behave just like an evolution thorn. While scheduling in PreStep is not perfectly equivalent to a real evolution, it would seem to cause fewer problems since only routines in MoL_PreStep and the ones inside of MoL's substep loop see anything different (namely the updated variables). If Exact itself moves to PseudoEvolution then one needs extra schedue statements to enforce the proper ordering.
Ian, Roland,
thanks for all the suggestions. I'm looking at the use of PseudoEvolution in my source tree and I see that some "analysis" functions are also scheduled in it (I looking, for instance, at ML_ADMConstraints). That, I guess, is one of the cases where scheduling Exact__initialize in this bin, without adding further constraints in the other functions scheduled here, would lead to incorrect behavior (Roland, do you have other examples?).
I would have thought, however, that any function that assumes that the evolution step has completed (such as one calculating the constraints) should be held until after pseudoevolution has completed. So PseudoEvolution does seem like the right place to schedule Exact__initialize, and if it breaks the tests it's only because of functions which shouldn't belong to this bin anyway. Any comments?
I don't think MoL_PseudoEvolution is the right schedule bin for Exact, for two reasons:
- when thorn Exact is used to evolve, thorn MoL will not be active, hence the bin won't be there - as someone (Eloisa?) mentioned, other thorns in MoL_PseudoEvolution assume that the main evolution (providing the spacetime and/or hydro) is done, so that they can access the ADM variables there
The first point could be solved by moving PseudoEvolution to a different thorn, e.g. a driver, or to the flesh itself.
The second point could (and maybe should) be solved by having markers in PseudoEvolution that state when the ADM and/or hydro variables are available, and always schedule your analysis routines with respect to these markers. (This would mimic SphericalSurface.)
At the moment, I would schedule Exact in EVOL (or PRESTEP).
-erik
On Tue, May 04, 2010 at 05:06:41PM -0500, Erik Schnetter wrote:
- when thorn Exact is used to evolve, thorn MoL will not be active,
hence the bin won't be there
MoL might not be active for the spacetime evolution. I could imagine it being active to hydro evolution with the spacetime being evolved by Exact. Quite a corner case, but probably possible.
At the moment, I would schedule Exact in EVOL (or PRESTEP).
I agree. If Exact is doing the evolution, it should be scheduled in EVOL.
Frank
Erik Schnetter ha scritto:
I don't think MoL_PseudoEvolution is the right schedule bin for Exact, for two reasons:
- when thorn Exact is used to evolve, thorn MoL will not be active,
hence the bin won't be there
Yes, one would have to have MoL activated anyway in order for this to work (which may not have to be ad hoc, as Frank says).
- as someone (Eloisa?) mentioned, other thorns in MoL_PseudoEvolution
assume that the main evolution (providing the spacetime and/or hydro) is done, so that they can access the ADM variables there
But is this actually correct? Roland's example with "BSSNtoADM" being naturally scheduled in PseudoEvolution makes it clear that there are at least two levels of pseudo-evolution: one that translates the gravitational/hydro variables between all the different formulations one may need, and a later one where these variables (cast in whichever formulation) are used to build derived objects. Also from Roland's comments, I understand that the latter class must remain in PseudoEvolution (rather than going in ANALYSIS which I would have found more natural) because derivatives calculated in ANALYSIS will be wrong at the refinement boundaries (did I get this point right?). Is there a way to solve this other problem perhaps?
The first point could be solved by moving PseudoEvolution to a different thorn, e.g. a driver, or to the flesh itself.
That would be very useful.
The second point could (and maybe should) be solved by having markers in PseudoEvolution that state when the ADM and/or hydro variables are available, and always schedule your analysis routines with respect to these markers. (This would mimic SphericalSurface.)
I feel like this should have been what PseudoEvolution was meant to accomplish...
At the moment, I would schedule Exact in EVOL (or PRESTEP).
PRESTEP is what is currently used, and leads to uninitialized points at the refinement boundaries (even when using the thorn to set all the ADM variables, with MoL not active). I guess EVOL is the next candidate; I'll give it a try and post back with the results.
Thanks! Eloisa
On May 4, 2010, at 21:37 , Eloisa Bentivegna wrote:
PRESTEP is what is currently used, and leads to uninitialized points at the refinement boundaries (even when using the thorn to set all the ADM variables, with MoL not active). I guess EVOL is the next candidate; I'll give it a try and post back with the results.
This is probably a different scheduling problem. Boundary conditions need to be applied after regridding and after restricting; maybe this is missing?
Exact should probably be scheduled at postinitial (or some other initial bin) postregridinitial postrestrict postrestrictinitial post_recover_variables
... essentially in all the places where MoL_PseudoEvolution is also scheduled.
Prestep and evol are scheduled right one after the other; scheduling in these bins makes no difference, except that everything in evol is scheduled after everything in prestep, which may help certain evolution thorns. I think prestep may be better for Exact, since some thorns schedule "in evol after MoL_Evolution", and this would then not necessarily be scheduled after Exact.
-erik
Hello Eloisa,
But is this actually correct? Roland's example with "BSSNtoADM" being naturally scheduled in PseudoEvolution makes it clear that there are at least two levels of pseudo-evolution: one that translates the gravitational/hydro variables between all the different formulations one may need, and a later one where these variables (cast in whichever formulation) are used to build derived objects. Also from Roland's comments, I understand that the latter class must remain in PseudoEvolution (rather than going in ANALYSIS which I would have found more natural) because derivatives calculated in ANALYSIS will be wrong at the refinement boundaries (did I get this point right?).
Yes this is my understanding. There was some discussion about this wrt to WeylScal4 (between Tanja, Erik, Ian, Peter, myself, around 2009-09-18). Essentially in ANALYSIS you get:
--8<-- What I wanted to say is that if the Psi4 grid function is set during ANALYSIS then the fine and coarse grid levels are not set in the proper order. To test this I wrote a tiny thorn with two functions that are both scheduled in ANALYSIS, one which sets a grid function to cctk_iteration+lvl/10, where lvl is the current refinement level and a second function to write one value from this grid function to stdout (see attachement). The output of those (for a six level system) looks like this:
# iteration i*j*k lvl grid-function debug: 0 0 0 0.000000 debug: 0 0 1 0.100000 debug: 0 0 2 0.200000 debug: 0 0 3 0.300000 debug: 0 0 4 0.400000 debug: 0 0 5 0.500000 debug: 0 0 6 0.600000 debug: 1 0 6 1.600000 <- debug: 2 0 5 2.500000 debug: 2 0 6 2.600000
Those ones with iteration number 0 are just initial data and are fine. If prolongation was to be used in ANALYSIS and it works as in EVOL then in step 1 (marked with an <- above) on level 6 one would need values from step 2 of level 5 which at this point have not yet been calculated, resulting in poison (or old values) being prolongated into the slice. --8<-- Essentially the time interpolation part of prolongation fails during ANALYSIS (so computing only every_coarse is almost fine [no restriction from fine to coarse happens]).
Is there a way to solve this other problem perhaps?
No idea.
Yours, Roland
users@lists.einsteintoolkit.org
-
Eloisa Bentivegna -
Erik Schnetter -
Frank Loeffler -
Ian Hinder -
Roland Haas