<div dir="ltr"><div style="font-family:arial,sans-serif;font-size:13px">Dear ET folks,</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">After months of work, I am pleased to release my rewrite of the Illinois numerical relativity group's staggered A-field GRMHD thorn to the ET community. The new thorn is called IllinoisGRMHD, and yields results identical (within roundoff error) to the original GRMHD code of the Illinois group with AMR & unigrids, with bitant symmetry & without, and with single or multiple MPI processes. We used precisely these GRMHD algorithms to produce our latest GRMHD-based papers.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">IllinoisGRMHD is written entirely in C, except for some C++ file I/O for debugging purposes, and all loops are OpenMP-compatible. The coding standard is C99.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px"><div>I would like to thank Frank and the CCT folks, who have graciously agreed to host IllinoisGRMHD on the ET SVN server, as the thorn awaits maintainers' approval for incorporation into the official ET. </div>
<div><br></div></div><div style="font-family:arial,sans-serif;font-size:13px">Download IllinoisGRMHD via the following command:</div><div style="font-family:arial,sans-serif;font-size:13px">svn co <a href="https://svn.einsteintoolkit.org/cactus/EinsteinEvolve/IllinoisGRMHD/trunk/" target="_blank">https://svn.einsteintoolkit.org/cactus/EinsteinEvolve/IllinoisGRMHD/trunk/</a></div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Browse IllinoisGRMHD source code in your web browser via:</div><div style="font-family:arial,sans-serif;font-size:13px">
<a href="https://svn.einsteintoolkit.org/cactus/EinsteinEvolve/IllinoisGRMHD/trunk/" target="_blank">https://svn.einsteintoolkit.org/cactus/EinsteinEvolve/IllinoisGRMHD/trunk/</a></div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">** Background **</div><div style="font-family:arial,sans-serif;font-size:13px">Roughly a decade of work has gone into the original code, and nearly all of the credit for debugging that battle-hardened code goes to Stu Shapiro and other former & current members of his group, to whom I give my great thanks and send the following message: Despite weeks entirely devoted to debugging this rewritten code, I found no major bugs in the original code.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">** License **</div><div style="font-family:arial,sans-serif;font-size:13px">The IllinoisGRMHD thorn is hereby released under the GNU GPL v2 or higher, to maximize compatibility with the ET thorns. </div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">** Changes from Original Code **</div><div style="font-family:arial,sans-serif;font-size:13px">The IllinoisGRMHD thorn contains significantly fewer lines than the code on which it is based, for two reasons:</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">1) Functionality that is no longer used (mostly dead code at this point) has been removed, and </div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">2) New data structures in the code make it more compact, reducing the total number of lines in the reconstruction & MHD RHS evaluation routines to ~1200. I hope that the data structures in the new code will make it easier to read, less intimidating for beginners, and much easier to modify & extend, to do new science.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">I started development of IllinoisGRMHD as a standalone code, to minimize the turnaround time in the first round of debugging and to try new ideas for enhancing the performance, as well as making the code more compact and easier to read. The standalone driver routines still exist, but bugfixes found in the "thorn version" have not yet been backported to these routines. I am happy to report that the new code is significantly faster than the original code. </div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">** Conservative-to-Primitives Routines **</div><div style="font-family:arial,sans-serif;font-size:13px">
We previously released our Conservative-to-Primitives routines to the ET mailing list, based on the very nice 2D HARM primitive solver of Scott Noble et al. An updated version is included in this release. </div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">** Future Work **</div><div style="font-family:arial,sans-serif;font-size:13px">Now that the IllinoisGRMHD thorn has been released, there is still some work to do, summarized below.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">1) If using AMR grids, this thorn requires support for staggered prolongation & restriction algorithms within Carpet. We open-sourced these algorithms for an old version of Carpet back in March of 2012. Since then, Carpet has moved forward and these original algorithms (which we still use in our old version of Carpet) are no longer compatible with the latest ET. Philipp Moesta informed me </div>
<div style="font-family:arial,sans-serif;font-size:13px">over the summer that he has ported these operators to a more recent Carpet, and I will be working with him in the coming weeks to incorporate these into the latest Carpet. </div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">2) This thorn currently exists as a drop-in replacement for the original GRMHD thorn of the Illinois group. I will be working in the coming weeks to make it more like a drop-in replacement for GRHydro instead, though perhaps someday it could be incorporated into GRHydro.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">3) There are several minor to-do tasks remaining in the code, marked within comments labeled "FIXME" (grep for it). </div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">4) There exist several possible avenues for code optimization, such as some possibly extraneous SYNC's within the schedule.ccl file.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">5) More and better comments that clarify what the code is doing will be incorporated.</div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">6) I am unsure how strongly this code violates coding standards & practices. E.g., I sometimes break lines according to the width of my laptop screen. Also, I love variable & function names that explain *exactly* what they do, which for some (possibly good) reason is frowned upon by other C/C++ programmers. I personally find that long names make it easier to read & understand a code years later. </div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">7) I apologize for the lack of full written documentation at this point, but promise that it will be forthcoming. In the meantime, I encourage you to look at the large number of comments in the code, and if you find a point of confusion, please email me, and I'll fix it! </div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">-Zach</div></div>