[Users] IllinoisGRMHD Code Compatibility Layer

[Zach Etienne]

Hi all,

Using Carpet/CarpetLib as an example, I was able to figure out how to
remove the nasty symlinks between IllinoisGRMHD and the
IllinoisGRMHD_CompatibilityLayer thorns (see latest commits).

Special thanks to Erik for pointing me in the right direction!

Now to address Erik's questions:

On Mon, Aug 17, 2015 at 11:10 PM, Erik Schnetter <schnetter at cct.lsu.edu>
wrote:

> On Mon, Aug 17, 2015 at 2:23 PM, Zach Etienne <zachetie at gmail.com> wrote:
>
>> Hi Erik,
>>
>> Great questions.
>>
>> - Is it necessary to have an explicit conversion thorn? Couldn't these
>> routines be simply part of IllinoisGRMHD? For example, McLachlan uses
>> its own variable, and contains its own routines to convert from/to ADMBase.
>>
>> IllinoisGRMHD is designed to maximize user-friendliness, and to this end
>> one of my goals is to minimize the amount of source code inside
>> IllinoisGRMHD that is not directly related to GRMHD evolution. Thus if
>> possible, I would prefer not incorporating functionality from these thorns
>> into IllinoisGRMHD.
>>
>
> That's fine. However, what I then don't understand -- why is the same
> function apparently needed both inside and outside of IllinoisGRMHD?
>

There are two functions that are needed inside & outside IllinoisGRMHD:
Function A) A function that computes the BSSN variables from the ADMBase
variables.
Function B) A function that checks the physicality of the IllinoisGRMHD
primitive variables and then computes the IllinoisGRMHD conservative
variables from the IllinoisGRMHD primitive variables (prim2con).

To give you a better understanding of why these functions are needed both
inside & outside IllinoisGRMHD, here is the order of operations OUTSIDE
IllinoisGRMHD within the initial data converter function (in the
compatibility layer thorn ID_converter_ILGRMHD). Note that when this thorn
is called, only HydroBase and ADMBase variables have been set.
1) [Function A]
2) Convert the HydroBase primitive variables (Valencia formulation) into
IllinoisGRMHD primitive variables.
3) Compute B from A (depends on Function A)
4) [Function B] (depends on Function A)

[Function A] is needed in IllinoisGRMHD in multiple places, to use the
latest, updated ADMBase variables during an evolution.
[Function B] is needed in IllinoisGRMHD's con2prim, so that if the
primitive or conservative variables fall outside the known physical range,
we can then compute prim2con so the primitives & conservatives are
consistent.



>
> - If the routines really need to exist in two thorns, then I suggest
>> having the actual routines in only one place, and calling the routines from
>> the other place. (I probably completely missed the original motivation.) If
>> necessary, you would need to introduce a third thorn containing these
>> routines that these two thorns can call.
>>
>> I would prefer not adding another thorn if possible, as I want to
>> minimize the effort required by a new user to read and understand the
>> IllinoisGRMHD evolution code. If they need to hunt in other thorns for
>> needed functions, learning IllinoisGRMHD would require more effort. I could
>> also raise a performance objection, in that the compiler may not be able to
>> optimize as well if functions deep within loops exist in other thorns.
>>
>> Here's a promising alternative: I just looked at the two
>> compatibility-layer thorns in a bit more detail. Turns out there are a
>> total of 2 functions from within IllinoisGRMHD that are needed by these two
>> compatibility-layer thorns. If I make these two functions share-able in
>> IllinoisGRMHD/interface.ccl, they can be accessed from the other thorns,
>> right? Two complications:
>> 1) these functions depend on a number of other functions within
>> IllinoisGRMHD, so would this linkage work?
>> 2) there is a header file "IllinoisGRMHD_headers.h" that is needed by
>> both thorns...
>>
>
> Cactus has a mechanism for this. This will just be a regular function
> call, so the fact that these two functions call other functions doesn't
> matter. If you are using a header file, then you don't need to additionally
> declare the function in interface.ccl. The interface.ccl mechanism is only
> needed if you want flexibility, so that one can change the thorns that
> provide these functions at run time. That doesn't seem relevant here.
>

Thanks to your suggestion, I figured this out. No symlinks necessary
anymore. Thanks!


>
>
>> In the end, as described in the IllinoisGRMHD trac ticket (
>> https://trac.einsteintoolkit.org/ticket/1796) I would like to move
>> IllinoisGRMHD to a wider arrangement (called WVUThorns, for example),
>> because it will be the first of many thorns my research team and I
>> intend to contribute to the Toolkit. So if all these IllinoisGRMHD-related
>> thorns exist in the same arrangement, maybe symbolic links aren't the worst
>> possible option. git seems to have no problem handling them.
>>
>
> As I mentioned, symbolic links give you the same file, and hence the same
> function name multiple times. This is not allowed by the linker. It is also
> confusing for people to see the same file / functions in two different
> places. Please try to find another solution. Regular function calls to
> another thorn will just work (if you provide respective header files), and
> there is no overhead associated with this.
>
> - Would scheduling "IN initial AFTER HydroBase_Initial" work?
>>
>> Not sure; I will need to try this.
>>
>
I have still not yet had a chance to try this. As a stopgap, I have looked
over a few initial data schedule.ccl files and
modified ID_converter_ILGRMHD's schedule.ccl file to be appropriately
scheduled after those thorns have completed setting up initial data.


>
>> - Similarly, there should be a group HydroBase_SetHydroBaseVars where the
>> respective routine should be scheduled. I think half of the functionality
>> that is present in ADMBase is still missing from HydroBase since, so far,
>> there was only one thorn (... or two very similar thorns ...) that used
>> HydroBase. Feel free to suggest the respective extensions to HydroBase,
>> modelled after ADMBase if possible.
>>
>> This is a neat idea, and brings up another item on my TODO list:
>> HydroBase currently does not allow for any input as to the staggering of
>> the A-fields. Currently IllinoisGRMHD must define its own A-field variables
>> in individual variable groups because the prolongation type for Avec in
>> HydroBase is not independently specifiable for the different components of
>> A. In addition to adding a group HydroBase_SetHydroBaseVars, it would
>> also be nice to generalize the specification of A-fields.
>>
>
> Can you make a concrete suggestion?
>

Given the magnitude of my proposed commit into the ET at the moment
(IllinoisGRMHD and compatibility layer thorns) and my very limited time now
that the semester has started, I would like to postpone this discussion
until after the next ET release. Currently I have a private thorn
"seed_magnetic_fields", which ignores HydroBase variables altogether when
setting (staggered) A fields. Unless a better thorn exists for setting up
generic magnetic field configurations from the vector potential, it would
be nice to extend this thorn to use unstaggered A fields as well. I will
release this thorn to the wild once IllinoisGRMHD & related thorns are
accepted into the ET.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20150824/a642273b/attachment.html