Dear ET folks,
This email is to continue the discussion we started during our seminar earlier today. The main reason we could not comment about collaborating more closely with ET is because we had not discussed this previously and furthermore not all of us were present on the conference call. (We have agreed that all of us should give input before making big decisions, such as contributing our code.) Having now discussed the issue with everyone, we are all very keen to contribute to the ET project where we can, as its success will be useful for our own future work, much as the basic Cactus-Carpet infrastructure has proven helpful in the past.
Here is our current thinking:
* Background
As you know, our latest GRMHD code evolves the vector potential A. The vector potential prescription we use carefully staggers all four components of the vector potential A, so that in unigrid the magnetic field B^i evolution is _equivalent_ to the standard, staggered constrained-transport scheme. A-field evolutions enable us to use any interpolation scheme we like at AMR refinement boundaries. However, there is a subtlety: Carpet does not have built-in prolongation/restriction operators for these staggered gridfunctions.
* Prolongation/Restriction Modifications to Carpet
To handle this problem, we modified some of the existing Carpet Lagrange prolongation/restriction code to instead interpolate the gridfunctions at the appropriate staggered gridpoints. Each of the four A-field components has its own staggering, so there are eight new pieces of code -- one for prolongating & restricting each component of A. We also have multiple interpolation schemes coded up, and each one may be switched on by a #define statement.
In terms of infrastructure, the rest is pretty much bookkeeping; each A-field component is stored as any other gridfunction, but we must be careful when using A-field information because e.g., Ax[i,j,k] on the grid actually stores Ax[i,j+1/2,k+1/2].
* Contributing to ET
Though it may not be compliant with the ET coding standards (e.g., we only handle the "double" data type), our changes to the Carpet infrastructure have been thoroughly tested, and we would be willing to contribute this code to the ET SVN. In return, we would like to work more closely with those of you who have access to the multipatch "cubed-spheres" code. Would you be willing to share this code with us?
Sincerely, The Illinois Numerical Relativity Group -Zach Etienne -Yuk Tung Liu -Vasilis Paschalidis -Stu Shapiro