[Users] Help needed in recovering from iadvertent hdf5 update on mac

[Ian Hinder]

On 3 Jun 2015, at 22:04, Comer Duncan <comer.duncan at gmail.com> wrote:

> Hi Erik,
> 
> Thanks for the perspective. Today after getting my macports back in shape, I have downloaded a new(?) the development ET and have built an executable including CT_MultiLevel. I've also run a couple of tests and all seems to be ok now.
> 
> Concerning the development version, when will it be possible to do git pull to get the latest dev version?  I use and contribute to one or two other software projects that use github and would hope since ET has or is moving to git that the usual methods of cloning various branches would be standardly available.  So, what can you say about when this will be possible from bitbucket.org?

Hi Comer,

As Erik mentioned, the ET is not stored in a single repository, because it is a modular system made up of components from many different sources.  For example, people need to be able to use Cactus without using the ET.  If we make a single monolithic ET repository, and put Cactus into it, then any changes to Cactus in the ET repository would not automatically go back into the Cactus repository.  Further, the ET contains some subversion repositories for historical and technical reasons, which is another barrier to having a single-git-repository solution.

The "official" way to check out the ET is the GetComponents tool.  Being able to switch branches within a single tree could be implemented in that tool, though I suspect it would need to be done carefully, as there are several things to consider (local changes, local commits, one repository being on a different branch than another, etc).

One way to handle a framework containing multiple independently-developed modules with git is to use a repository which contains only pointers to other repositories, known as "submodules".  This is often used in other open-source repositories to include required libraries with the code.  It is also used heavily by Android developers.  Barry Wardell and I have set up such a system for the ET.  Unfortunately, it is not a completely smooth solution, as submodules have a reputation for being tricky to work with.  As long as you only want to do simple things, you just need a few commands, and things will work fairly well.  But the system gives you a lot of power that you can use to shoot yourself in the foot!  We work around the subversion issue by maintaining git mirrors of the subversion repositories.

To check out the ET master branch and all its submodules:

	git clone --recursive https://bitbucket.org/einsteintoolkit/einsteintoolkit.git Cactus

If you later need to update to the latest version,

	cd Cactus
	git pull
	git submodule update

To check out a release branch, you can use

	git checkout ET_2015_05
	git submodule update

To go back to master,

	git checkout master
	git submodule update


That covers most of the common operations that you will need to do if you are just using the code, rather than developing it.

Any modifications to files that you may have made will be preserved on switching branches, unless there is a conflict with a change between the branches, in which case you will get a conflict warning when running git submodule update.

The top-level einsteintoolkit repository is known as a super-repository, since it contains pointers to other repositories, which have been checked out under the super-repository in your working copy.  So operations on the super-repository like those above should be carried out in, for example, the top-level Cactus directory, not within a submodule directory, or they will refer to the submodule repository instead.  A specific commit in the super-repository contains pointers to specific commits in the submodules.  You can see the history of the super-repository with

	git log

The different commits shown there represent historical states of the super-repository; snapshots in time of the submodules.  You can check out an old version:

	git checkout 468de39937fbb970a3077b97b6c969189255a363
	git submodule update

The "git submodule update" is necessary because the checkout doesn't alter the checked-out versions in the submodules.  You need to do that explicitly.

The BitBucket super-repository is updated automatically whenever the component repositories are pushed to, but there may be a delay of a few minutes, as it takes a while to scan for all changes, and the server we run this process on is quite old.  

One reason I like this system is that it makes it very easy to go back to a previous state.  I am no longer afraid of updating to the latest version of the ET, because I know I can always easily go back to what I had before.  I can also easily see which submodules have local changes in their working copies,

	git status

and what those changes are,

	git submodule foreach git diff

I can see when I have local commits to the submodules which are not in my copy of the super-repository, or vice-versa,

	git submodule -q summary

Each submodule (you can get a list with git submodule status) is its own repository, so if you cd into it, you can do all the normal git things.  [One annoying feature is that the submodules are checked out into "detached head" state, rather than being on a particular branch. If you do want to commit changes in a submodule, you probably want to create your own branch in the submodule first with "git checkout -b mychanges".]

The most powerful feature of all, I think, is that I can identify a complete revision of the code by a single git hash, though this feature is not yet so useful, because you also have to make sure that you store a  copy of the super-repository that the hash was made in, and all the submodules with the corresponding commits.  

Feel free to use this super-repository if you like; we have been using it for several years, and keep it up-to-date.

-- 
Ian Hinder
http://members.aei.mpg.de/ianhin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20150604/ce7bb478/attachment-0001.html