<div dir="ltr">Hi all,<div class="gmail_extra"><br></div><div class="gmail_extra">Using Carpet/CarpetLib as an example, I was able to figure out how to remove the nasty symlinks between IllinoisGRMHD and the IllinoisGRMHD_CompatibilityLayer thorns (see latest commits). </div><div class="gmail_extra"><br></div><div class="gmail_extra">Special thanks to Erik for pointing me in the right direction!</div><div class="gmail_extra"><br></div><div class="gmail_extra">Now to address Erik's questions:</div><div class="gmail_extra">
<br><div class="gmail_quote">On Mon, Aug 17, 2015 at 11:10 PM, Erik Schnetter <span dir="ltr"><<a href="mailto:schnetter@cct.lsu.edu" target="_blank">schnetter@cct.lsu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span class="">On Mon, Aug 17, 2015 at 2:23 PM, Zach Etienne <span dir="ltr"><<a href="mailto:zachetie@gmail.com" target="_blank">zachetie@gmail.com</a>></span> wrote:<br></span><div class="gmail_extra"><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Erik,<div><br></div><div>Great questions.</div><span><div><br></div><div><div style="font-size:12.8000001907349px">- Is it necessary to have an explicit conversion thorn? Couldn't these routines be simply part of <span style="font-size:13px">IllinoisGRMHD? For example, McLachlan uses its own variable, and contains its own routines to convert from/to ADMBase.</span></div></div><div><span style="font-size:13px"><br></span></div></span><div>IllinoisGRMHD is designed to maximize user-friendliness, and to this end one of my goals is to minimize the amount of source code inside IllinoisGRMHD that is not directly related to GRMHD evolution. Thus if possible, I would prefer not incorporating functionality from these thorns into IllinoisGRMHD.</div></div></blockquote><div><br></div></span><div>That's fine. However, what I then don't understand -- why is the same function apparently needed both inside and outside of IllinoisGRMHD?</div></div></div></div></blockquote><div><br></div><div>There are two functions that are needed inside & outside IllinoisGRMHD: </div><div>Function A) A function that computes the BSSN variables from the ADMBase variables.</div><div>Function B) A function that checks the physicality of the IllinoisGRMHD primitive variables and then computes the IllinoisGRMHD conservative variables from the IllinoisGRMHD primitive variables (prim2con).</div><div><br></div><div>To give you a better understanding of why these functions are needed both inside & outside IllinoisGRMHD, here is the order of operations OUTSIDE IllinoisGRMHD within the initial data converter function (in the compatibility layer thorn ID_converter_ILGRMHD). Note that when this thorn is called, only HydroBase and ADMBase variables have been set.</div><div>1) [Function A]</div><div>2) Convert the HydroBase primitive variables (Valencia formulation) into IllinoisGRMHD primitive variables.</div><div>3) Compute B from A (depends on Function A)</div><div>4) [Function B] (depends on Function A)</div><div><br></div><div>[Function A] is needed in IllinoisGRMHD in multiple places, to use the latest, updated ADMBase variables during an evolution.</div><div>[Function B] is needed in IllinoisGRMHD's con2prim, so that if the primitive or conservative variables fall outside the known physical range, we can then compute prim2con so the primitives & conservatives are consistent.</div><div><br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span><div><span style="font-size:13px">- If the routines really need to exist in two thorns, then I suggest having the actual routines in only one place, and calling the routines from the other place. (I probably completely missed the original motivation.) If necessary, you would need to introduce a third thorn containing these routines that these two thorns can call.</span><br></div><div><span style="font-size:13px"><br></span></div></span><div><span style="font-size:13px">I would prefer not adding another thorn if possible, as I want to minimize the effort required by a new user to read and understand the IllinoisGRMHD evolution code. If they need to hunt in other thorns for needed functions, learning IllinoisGRMHD would require more effort. I could also raise a performance objection, in that the compiler may not be able to optimize as well if functions deep within loops exist in other thorns.</span></div><div><span style="font-size:13px"><br></span></div><div><span style="font-size:13px">Here's a promising alternative: I just looked at the two compatibility-layer thorns in a bit more detail. Turns out there are a total of 2 functions from within IllinoisGRMHD that are needed by these two compatibility-layer thorns. If I make these two functions share-able in IllinoisGRMHD/interface.ccl, they can be accessed from the other thorns, right? Two complications: </span></div><div><span style="font-size:13px">1) these functions depend on a number of other functions within IllinoisGRMHD, so would this linkage work?</span></div><div><span style="font-size:13px">2) there is a header file "</span>IllinoisGRMHD_headers.h"<span style="font-size:13px"> that is needed by both thorns...</span></div></div></blockquote><div><br></div></span><div>Cactus has a mechanism for this. This will just be a regular function call, so the fact that these two functions call other functions doesn't matter. If you are using a header file, then you don't need to additionally declare the function in interface.ccl. The interface.ccl mechanism is only needed if you want flexibility, so that one can change the thorns that provide these functions at run time. That doesn't seem relevant here.</div></div></div></div></blockquote><div><br></div><div>Thanks to your suggestion, I figured this out. No symlinks necessary anymore. Thanks!</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><span style="font-size:13px">In the end, as described in the IllinoisGRMHD trac ticket (</span><a href="https://trac.einsteintoolkit.org/ticket/1796" target="_blank">https://trac.einsteintoolkit.org/ticket/1796</a>)<span style="font-size:13px"> I would like to move IllinoisGRMHD to a wider arrangement (called WVUThorns, for example), because </span><span style="font-size:13px">it will be the first of many thorns my research team and I intend to contribute to the Toolkit. So if all these IllinoisGRMHD-related thorns exist in the same arrangement, maybe symbolic links aren't the worst possible option. git seems to have no problem handling them.</span></div></div></blockquote><div><br></div></span><div>As I mentioned, symbolic links give you the same file, and hence the same function name multiple times. This is not allowed by the linker. It is also confusing for people to see the same file / functions in two different places. Please try to find another solution. Regular function calls to another thorn will just work (if you provide respective header files), and there is no overhead associated with this.</div><span class=""><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span><div><span style="font-size:13px">- Would scheduling "IN initial AFTER HydroBase_Initial" work?</span><br></div><div><span style="font-size:13px"><br></span></div></span><div><span style="font-size:13px">Not sure; I will need to try this.</span></div></div></blockquote></span></div></div></div></blockquote><div><br></div><div>I have still not yet had a chance to try this. As a stopgap, I have looked over a few initial data schedule.ccl files and modified ID_converter_ILGRMHD's schedule.ccl file to be appropriately scheduled after those thorns have completed setting up initial data.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span><div><div style="font-size:12.8000001907349px"><span style="font-size:13px"><br>- Similarly, there should be a group HydroBase_SetHydroBaseVars where the respective routine should be scheduled. I think half of the functionality that is present in ADMBase is still missing from HydroBase since, so far, there was only one thorn (... or two very similar thorns ...) that used HydroBase. Feel free to suggest the respective extensions to HydroBase, modelled after ADMBase if possible.</span></div></div><div><span style="font-size:13px"><br></span></div></span><div><span style="font-size:13px">This is a neat idea, and brings up another item on my TODO list: HydroBase currently does not allow for any input as to the staggering of the A-fields. Currently IllinoisGRMHD must define its own A-field variables in individual variable groups because the prolongation type for Avec in HydroBase is not independently specifiable for the different components of A. In addition to adding a group </span><span style="font-size:13px">HydroBase_SetHydroBaseVars, i</span><span style="font-size:13px">t would also be nice to generalize the specification of A-fields.</span></div></div></blockquote><div><br></div></span><div>Can you make a concrete suggestion?</div></div></div></div></blockquote><div><br></div><div>Given the magnitude of my proposed commit into the ET at the moment (IllinoisGRMHD and compatibility layer thorns) and my very limited time now that the semester has started, I would like to postpone this discussion until after the next ET release. Currently I have a private thorn "seed_magnetic_fields", which ignores HydroBase variables altogether when setting (staggered) A fields. Unless a better thorn exists for setting up generic magnetic field configurations from the vector potential, it would be nice to extend this thorn to use unstaggered A fields as well. I will release this thorn to the wild once IllinoisGRMHD & related thorns are accepted into the ET.</div><div><br></div></div></div></div>