Comer

You should be able to clone Cactus and Einstein Toolkit arrangements on Bitbucket in the usual way, and submit pull requests at your leisure. We haven't quite set up our bug tracker yet for this new infrastructure, so at the moment we're also keeping track of bug reports and pull requests on trac.einsteintoolkit.org. I'd recommend opening a trac ticket for each non-trivial pull request, and have the strategic discussions there. If there's a need to dissect code line-by-line (which I hope isn't necessary too often), then I'm using Bitbucket's commenting system for this.

The one difference to other projects is that we decided that each arrangement should be its own repository. That may or may not have been a good idea, as people have to deal with too many repositories. However, having all of the Einstein Toolkit in a single repository is also not possible since different parts are contributed by different people with different licences etc., and we want to keep this distinction. Think of an arrangement as a "plug-in package" that lives in its on repository.

-erik

On Wed, Jun 3, 2015 at 4:04 PM, Comer Duncan comer.duncan@gmail.com wrote:

Hi Erik,

Thanks for the perspective. Today after getting my macports back in shape, I have downloaded a new(?) the development ET and have built an executable including CT_MultiLevel. I've also run a couple of tests and all seems to be ok now.

Concerning the development version, when will it be possible to do git pull to get the latest dev version? I use and contribute to one or two other software projects that use github and would hope since ET has or is moving to git that the usual methods of cloning various branches would be standardly available. So, what can you say about when this will be possible from bitbucket.org?

Thanks.

Comer

On Wed, Jun 3, 2015 at 11:10 AM, Erik Schnetter schnetter@cct.lsu.edu wrote:

...

Comer

HDF5 1.8.15 is broken. A patch will be released shortly. The development version of thorn ExternalLibraries/HDF5 already contains a copy of the patched version; if you use this version of the thorn and specify HDF5_DIR=BUILD in your option list, you should be fine.

-erik

On Wed, Jun 3, 2015 at 11:04 AM, Comer Duncan comer.duncan@gmail.com wrote:

...

Hi Ian,

I got back Monday night and yesterday decided to get CT_MultiLevel and rebuild so I can start learning how the multigrid stuff works with some examples. I got the CT_MultiLevel directories from bitbucket, copied it into place in EinsteinInitialdata. in the meantime I had done a weekly update of all my macports stuff (not a good idea!). I then proceeded to rebuild cactus. All went fine (the CT_MultiLevel seems to compile ok) until it got to iohdf5. It crashed complaining about iohdf5 stuff. I have pasted the build to http://pastebin.com/dqMtCL3A . Since the hdf5 stuff got updated I went through the procedure of deactivating and in some cases uninstalling hdf related stuff and believe I left it in the state in which no hdf5 is active. I have pasted a record of the various steps I went through as I discovered what depended on what and tried to get hdf5 unactivated. I believe I was successful. Please see the following paste: http://pastebin.com/4brgVdmE .

With the optionfile having BUILD as the selected option for HDF5 as was the case when I implemented the workaround last week that you suggested.

So now, I am back to a similar spot except that now I do not have mpi and/or gcc related issues. Also, I have the CT_MultiLevel stuff in place.

Can you suggest some next steps which can get be out of this hdf5 fix?

Thanks for the help!!!

Comer

---

Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users

-- Erik Schnetter schnetter@cct.lsu.edu http://www.perimeterinstitute.ca/personal/eschnetter/

Hi Roland,

Thanks very much. These are quite helpful!

Comer

On Thu, Jun 4, 2015 at 4:20 AM, Roland Haas rhaas@aei.mpg.de wrote:

Hello Comer,

...

Concerning the development version, when will it be possible to do git

pull

...

to get the latest dev version? I use and contribute to one or two other software projects that use github and would hope since ET has or is

moving

...

to git that the usual methods of cloning various branches would be standardly available. So, what can you say about when this will be possible from bitbucket.org?

In case you only want to update a single part of the checkout to the development version, eg. one of the ExternalLibraries (they are still in subversion and will likely stay because git does not handle large binary blobs of data like the tarballs very well) you can do:

cd arrangements/ExternalLibraries/HDF5/ svn switch '^/trunk' svn update

and for git based repos

cd repos/flesh git checkout master git pull

you can use the regular git pull and svn update commands in the repositories. There is also an update (-u) option to GetComponents however it may switch branches again since the thornlist specified the release branch.

Yours, Roland

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net. _______________________________________________ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users

Hi Ian,

Wow! This sounds like it will take a bit to adjust to the supermodule way of life, but it sounds very useful. I appreciate your comments and explanations.

Comer

On Thu, Jun 4, 2015 at 4:49 AM, Ian Hinder ian.hinder@aei.mpg.de wrote:

On 3 Jun 2015, at 22:04, Comer Duncan comer.duncan@gmail.com wrote:

Hi Erik,

Thanks for the perspective. Today after getting my macports back in shape, I have downloaded a new(?) the development ET and have built an executable including CT_MultiLevel. I've also run a couple of tests and all seems to be ok now.

Concerning the development version, when will it be possible to do git pull to get the latest dev version? I use and contribute to one or two other software projects that use github and would hope since ET has or is moving to git that the usual methods of cloning various branches would be standardly available. So, what can you say about when this will be possible from bitbucket.org?

Hi Comer,

As Erik mentioned, the ET is not stored in a single repository, because it is a modular system made up of components from many different sources. For example, people need to be able to use Cactus without using the ET. If we make a single monolithic ET repository, and put Cactus into it, then any changes to Cactus in the ET repository would not automatically go back into the Cactus repository. Further, the ET contains some subversion repositories for historical and technical reasons, which is another barrier to having a single-git-repository solution.

The "official" way to check out the ET is the GetComponents tool. Being able to switch branches within a single tree could be implemented in that tool, though I suspect it would need to be done carefully, as there are several things to consider (local changes, local commits, one repository being on a different branch than another, etc).

One way to handle a framework containing multiple independently-developed modules with git is to use a repository which contains only pointers to other repositories, known as "submodules". This is often used in other open-source repositories to include required libraries with the code. It is also used heavily by Android developers. Barry Wardell and I have set up such a system for the ET. Unfortunately, it is not a completely smooth solution, as submodules have a reputation for being tricky to work with. As long as you only want to do simple things, you just need a few commands, and things will work fairly well. But the system gives you a lot of power that you can use to shoot yourself in the foot! We work around the subversion issue by maintaining git mirrors of the subversion repositories.

To check out the ET master branch and all its submodules:

git clone --recursive https://bitbucket.org/einsteintoolkit/einsteintoolkit.git Cactus

If you later need to update to the latest version,

cd Cactus git pull git submodule update

To check out a release branch, you can use

git checkout ET_2015_05 git submodule update

To go back to master,

git checkout master git submodule update

That covers most of the common operations that you will need to do if you are just using the code, rather than developing it.

Any modifications to files that you may have made will be preserved on switching branches, unless there is a conflict with a change between the branches, in which case you will get a conflict warning when running git submodule update.

The top-level einsteintoolkit repository is known as a super-repository, since it contains pointers to other repositories, which have been checked out under the super-repository in your working copy. So operations on the super-repository like those above should be carried out in, for example, the top-level Cactus directory, not within a submodule directory, or they will refer to the submodule repository instead. A specific commit in the super-repository contains pointers to specific commits in the submodules. You can see the history of the super-repository with

git log

The different commits shown there represent historical states of the super-repository; snapshots in time of the submodules. You can check out an old version:

git checkout 468de39937fbb970a3077b97b6c969189255a363 git submodule update

The "git submodule update" is necessary because the checkout doesn't alter the checked-out versions in the submodules. You need to do that explicitly.

The BitBucket super-repository is updated automatically whenever the component repositories are pushed to, but there may be a delay of a few minutes, as it takes a while to scan for all changes, and the server we run this process on is quite old.

One reason I like this system is that it makes it very easy to go back to a previous state. I am no longer afraid of updating to the latest version of the ET, because I know I can always easily go back to what I had before. I can also easily see which submodules have local changes in their working copies,

git status

and what those changes are,

git submodule foreach git diff

I can see when I have local commits to the submodules which are not in my copy of the super-repository, or vice-versa,

git submodule -q summary

Each submodule (you can get a list with git submodule status) is its own repository, so if you cd into it, you can do all the normal git things. [One annoying feature is that the submodules are checked out into "detached head" state, rather than being on a particular branch. If you do want to commit changes in a submodule, you probably want to create your own branch in the submodule first with "git checkout -b mychanges".]

The most powerful feature of all, I think, is that I can identify a complete revision of the code by a single git hash, though this feature is not yet so useful, because you also have to make sure that you store a copy of the super-repository that the hash was made in, and all the submodules with the corresponding commits.

Feel free to use this super-repository if you like; we have been using it for several years, and keep it up-to-date.

-- Ian Hinder http://members.aei.mpg.de/ianhin