#620: Simplify timelevel handling for Cactus thorn writers -------------------------+-------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: new Priority: major | Milestone: Component: Cactus | Version: Keywords: | -------------------------+-------------------------------------------------- Currently, a thorn writer must declare the maximum number of timelevels for a variable in the interface.ccl file, then allocate a specific number of timelevels for which storage is allocated in the schedule.ccl file. There are rules for how many timelevels a variable should have. For example, to evolve using MoL I think you need at least two, whereas to evolve using mesh refinement with time prolongation order 2 you need three.
The problem is that the person writing the thorn shouldn't have to know how it is going to be used. If I write an evolution thorn, I should not care whether it is used with mesh refinement or not. I certainly shouldn't have to care what prolongation order is going to be used.
Would it be possible for Cactus to automatically determine the number of timelevels needed for a given variable? My proposal is that it should not be necessary for the user to specify the number of timelevels in the interface.ccl or schedule.ccl file for variables declared in that thorn. The only time a thorn writer should have to do that is if that thorn specifically uses the other timelevels. For example, a thorn which couples to MoL will probably only ever read or write to the current timelevel. MoL, on the other hand, could tell Cactus that it needs a certain number of timelevels at runtime for those variables. Similarly for mesh refinement.
This goes along with the planned changes to the scheduling system to make it easier to program Cactus thorns and make it harder to make errors.
#620: Simplify timelevel handling for Cactus thorn writers --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: new Priority: major | Milestone: Component: Cactus | Version: Resolution: | Keywords: --------------------------+-------------------------------------------------
Comment (by eschnett):
Each timelevel is stored in its own variable, e.g. gxx, gxx_p, gxx_p_p, etc. The number of timelevels declared in interface.ccl defines how many of these variables will be passed into each subroutine, and thus needs to be known at build time.
The number of timelevels that is allocated is much more flexible; this is the number specified in the schedule.ccl file.
There is, in principle, nothing wrong with allocating more timelevels than there are variables. The additional timelevels could not be accessed in "straightforward" code, but are still visible e.g. via CCTK_VarDataPtr.
That is, it should be fairly easy to do away with the maximum number of timelevels. The number of timelevels declared in the interface.ccl becomes the number of timelevels directly accessible in the code -- typically only 1 is necessary, except if past timelvels are initialised explicitly, which is rarely a convenient way to do so.
#620: Simplify timelevel handling for Cactus thorn writers --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: new Priority: major | Milestone: Component: Cactus | Version: Resolution: | Keywords: --------------------------+-------------------------------------------------
Comment (by eschnett):
I believe this could be implemented in the following way:
1. In addition to the flesh function MaxTimelevels that returns the maximum number of timelevels, we add a SetMaxTimelevels that allows increasing the maximum number of timelevels for a group. We probably don't want to allow decreasing this number, because some code may cache this number (maybe implicitly in the CCTK_ARGUMENTS bindings). This would not actually allocate memory, it only increases the maximum. The flesh provides a data structure cGH->data which contains pointers to all timelevels of all grid variables; this data structure needs to be resized (and the new elements initialised) in this function.
2. The driver functions that modify the number of active timelevels call SetMaxTimelevels if necessary before they allocate storage. If a driver does not do this, the behaviour remains as is at the moment.
3. The drivers that allow increasing this number will also have to checkpoint and restore this value while checkpointing. It seems that the checkpointing data format does not need to be changed; increasing the maximum to the number of timelevels found in the checkpoint file seems fine.
This data structure is set up in CactusDefaultSetupGH (file Comm/CactusDefaultComm.c).
The maximum number of timelevels per group is stored in the field n_timelevels of struct cGroupDefinition, which is declared privately in main/Groups.c, and can be read via MaxTimelevels.
Does this sound reasonable?
#620: Simplify timelevel handling for Cactus thorn writers --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: review Priority: major | Milestone: Component: Cactus | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by eschnett):
* status: new => review
#620: Simplify timelevel handling for Cactus thorn writers --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: review Priority: major | Milestone: Component: Cactus | Version: Resolution: | Keywords: --------------------------+-------------------------------------------------
Comment (by knarf):
It does.
#620: Simplify timelevel handling for Cactus thorn writers --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: reopened Priority: major | Milestone: Component: Cactus | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by eschnett):
* status: review => reopened
Comment:
This is a good idea. Setting status to "open" since there is no patch yet.
#620: Simplify timelevel handling for Cactus thorn writers --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: reopened Priority: major | Milestone: Component: Cactus | Version: Resolution: | Keywords: --------------------------+-------------------------------------------------
Comment (by hinder):
That would address interface.ccl. What about schedule.ccl? I would like an application thorn to only have to declare the number of timelevels that it actually needs. So a wave equation thorn, writing to only one timelevel, would just say {{{ STORAGE: phi[1] }}} I think the 1 is actually optional in this case. Then we need a way for MoL to dynamically tell the driver that it needs two timelevels for phi, and Carpet to indicate that it needs 3 if the time prolongation order requires it. What would be a good way of implementing this? It is not quite the same as SetMaxTimelevels.
#620: Simplify timelevel handling for Cactus thorn writers --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: reopened Priority: major | Milestone: Component: Cactus | Version: Resolution: | Keywords: --------------------------+-------------------------------------------------
Comment (by hinder):
There is already a flesh function CCTK_GroupStorageIncrease (http://cactuscode.org/documentation/referencemanual/ReferenceManualch2.html#...). Maybe we can just add calls to this function to MoL and Carpet to increase the amount of storage when it is needed? This function would then call SetMaxTimelevels to ensure that the data structures had space for enough timelevels, as described above.
#620: Simplify timelevel handling for Cactus thorn writers --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: reopened Priority: major | Milestone: Component: Cactus | Version: Resolution: | Keywords: --------------------------+-------------------------------------------------
Comment (by eschnett):
The current mechanism for a thorn indicating that it needs time levels is to call cctk_GroupStorageIncrease, as this would allocate new time levels if necessary. This call will never reduce the number of time levels.
These calls need to occur late enough, so that the GH already exists and grid functions can have storage allocated, but also needs to occur early enough that initial data can still be set up. I believe basegrid would be a good time.
trac@lists.einsteintoolkit.org
-
Einstein Toolkit