Dear ET folks,
This email is to continue the discussion we started during our seminar earlier today. The main reason we could not comment about collaborating more closely with ET is because we had not discussed this previously and furthermore not all of us were present on the conference call. (We have agreed that all of us should give input before making big decisions, such as contributing our code.) Having now discussed the issue with everyone, we are all very keen to contribute to the ET project where we can, as its success will be useful for our own future work, much as the basic Cactus-Carpet infrastructure has proven helpful in the past.
Here is our current thinking:
* Background
As you know, our latest GRMHD code evolves the vector potential A. The vector potential prescription we use carefully staggers all four components of the vector potential A, so that in unigrid the magnetic field B^i evolution is _equivalent_ to the standard, staggered constrained-transport scheme. A-field evolutions enable us to use any interpolation scheme we like at AMR refinement boundaries. However, there is a subtlety: Carpet does not have built-in prolongation/restriction operators for these staggered gridfunctions.
* Prolongation/Restriction Modifications to Carpet
To handle this problem, we modified some of the existing Carpet Lagrange prolongation/restriction code to instead interpolate the gridfunctions at the appropriate staggered gridpoints. Each of the four A-field components has its own staggering, so there are eight new pieces of code -- one for prolongating & restricting each component of A. We also have multiple interpolation schemes coded up, and each one may be switched on by a #define statement.
In terms of infrastructure, the rest is pretty much bookkeeping; each A-field component is stored as any other gridfunction, but we must be careful when using A-field information because e.g., Ax[i,j,k] on the grid actually stores Ax[i,j+1/2,k+1/2].
* Contributing to ET
Though it may not be compliant with the ET coding standards (e.g., we only handle the "double" data type), our changes to the Carpet infrastructure have been thoroughly tested, and we would be willing to contribute this code to the ET SVN. In return, we would like to work more closely with those of you who have access to the multipatch "cubed-spheres" code. Would you be willing to share this code with us?
Sincerely, The Illinois Numerical Relativity Group -Zach Etienne -Yuk Tung Liu -Vasilis Paschalidis -Stu Shapiro
Dear Zach et al.,
thank you for your e-mail and your general willingness to contribute some of your code to the Einstein Toolkit.
This is an open discussion. Te opinions that I am voicing in this e-mail are my personal opinions and I am not speaking for the ET toolkit consortium nor the team of maintainers. Others among this team may have other opinions.
On Mon, Nov 28, 2011 at 04:28:14PM -0600, Zach Etienne wrote:
- Contributing to ET
Though it may not be compliant with the ET coding standards (e.g., we only handle the "double" data type), our changes to the Carpet infrastructure have been thoroughly tested, and we would be willing to contribute this code to the ET SVN. In return, we would like to work more closely with those of you who have access to the multipatch "cubed-spheres" code. Would you be willing to share this code with us?
Many people and groups have contributed codes and much time to the Einstein Toolkit project without receiving "a return" in the form of access to code that is currently not public and nor generally used or owned by the entire group of maintainers. I think it would be highly unfair to make such a "trade" with one particular group and I will personally not support such an agreement.
What you would get -- and what everybody gets who contributes -- is respect, pride, and acknowledgement (and citations) for contributing code to the Einstein Toolkit as open source. Should you decide to actively help with the Toolkit and its maintenance, you will also be able to co-author a future follow-up ET paper.
Moreover, we have just recently submitted a grant proposal to NSF for renewal of the grant supporting much of the Einstein Toolkit work done at LSU, Georgia Tech, RIT, and Caltech. Part of our proposal is to bring multi-block (cubed-sphere) grids and infrastructure as open source to the Einstein Toolkit. So, should we receive funding to carry out this research/development and should you decide to be actively involved in this, I think you would be more than welcome to have early access to the development version of the planned multi-block improvements.
Best regards,
- Christian Ott
On 29 Nov 2011, at 00:03, Christian D. Ott wrote:
What you would get -- and what everybody gets who contributes -- is respect, pride, and acknowledgement (and citations) for contributing code to the Einstein Toolkit as open source. Should you decide to actively help with the Toolkit and its maintenance, you will also be able to co-author a future follow-up ET paper.
You also get many eyes on your code, which may spot bugs that would be missed otherwise. Additionally, ET members ensure that the code builds and runs successfully on a large number of machines for each ET release. A recent example is the work done to make sure the PITT Null Code, a recent contribution, worked on a number of machines where it was failing. Further, as part of the ET, the community will ensure that the code remains compatible with any possible changes in Cactus or other thorns that it depends on. In other words, now that you have written the code, the community can help with the unglamorous "maintenance" that it requires.
Dear Christian, Ian, and others,
Sorry for the misunderstanding. We intended to supply our interpolation operators to ET without any preconditions. Note that the code we contribute is no silver bullet; there is a lot of careful bookkeeping that must be done to construct an MHD code around these infrastructure improvements. Our hope was that our contribution would bring us closer to those in the ET community who have multipatch code, which could be beneficial to us. Our idea is that we play an advisory role in helping the multipatch team with construction of such an MHD code, which would expedite the process greatly, and allow the multipatch code to be shared with us.
Sincerely,
The Illinois Numerical Relativity Group -Zach Etienne -Yuk Tung Liu -Vasilis Paschalidis -Stu Shapiro
On Mon, Nov 28, 2011 at 5:38 PM, Ian Hinder ian.hinder@aei.mpg.de wrote:
On 29 Nov 2011, at 00:03, Christian D. Ott wrote:
What you would get -- and what everybody gets who contributes -- is respect, pride, and acknowledgement (and citations) for contributing code to the Einstein Toolkit as open source. Should you decide to actively help with the Toolkit and its maintenance, you will also be able to co-author a future follow-up ET paper.
You also get many eyes on your code, which may spot bugs that would be missed otherwise. Additionally, ET members ensure that the code builds and runs successfully on a large number of machines for each ET release. A recent example is the work done to make sure the PITT Null Code, a recent contribution, worked on a number of machines where it was failing. Further, as part of the ET, the community will ensure that the code remains compatible with any possible changes in Cactus or other thorns that it depends on. In other words, now that you have written the code, the community can help with the unglamorous "maintenance" that it requires.
-- Ian Hinder http://numrel.aei.mpg.de/people/hinder
Hi Zach,
On Mon, Nov 28, 2011 at 04:28:14PM -0600, Zach Etienne wrote:
as contributing our code.) Having now discussed the issue with everyone, we are all very keen to contribute to the ET project where we can, as its success will be useful for our own future work, much as the basic Cactus-Carpet infrastructure has proven helpful in the past.
Regardless of everything else: are you interested in registering as ET users: http://einsteintoolkit.org/about/members/ ?
Though it may not be compliant with the ET coding standards (e.g., we only handle the "double" data type), our changes to the Carpet infrastructure have been thoroughly tested, and we would be willing to contribute this code to the ET SVN.
Please contact me directly for user accounts. You will need them to upload anything to the ET SVN, even to the 'incoming sandbox'.
In return, we would like to work more closely with those of you who have access to the multipatch "cubed-spheres" code. Would you be willing to share this code with us?
The Einstein Toolkit does not contain this code. If it would, it would be publicly available. As already mentioned, contributing to the Einstein Toolkit does not mean to get access to some other code in return. What you do get is acknowledgment and respect. Besides that, you get more eyes on your code: a good thing because in the long run because it means fewer errors and shared maintenance and porting work.
I would suggest to start this slowly. I am always happy to see contributions to the toolkit. The way this usually is handled is to make the code publicly available first. If you don't have a convenient way to do that we do provide one just for that purpose. With the code available, people have a better ground for discussion.
After especially the maintainers had a chance to look at the code they will decide about the inclusion into the ET, and quite often request or make changes before that.
Frank
users@lists.einsteintoolkit.org
-
Christian D. Ott -
Frank Loeffler -
Ian Hinder -
Zach Etienne