Present: Frank, Roland, Matt, Erik, Peter
using intel 14 compiler: * Frank tried and failed with a compiler abort inside of CTGamma * Erik tried on a machine with both gcc 4.8 and gcc 4.7 installed which causes the compilation to fail due to improperly chosen include files (see: http://software.intel.com/en-us/forums/topic/489013)
enabled-thorns option: * triggered by ticket #1532 * the current method is not yet optimal * several options discussed: 1. disable thorns from inside Cactus when they detect hey cannot run, this needs to be done very early in the build process 2. use two thornlists, one generated automatically from the other. Simfactory or GetComponents may do this. 3. thorns detect at configure time that they cannot compile and turn themselves into No-OP thorns. This shifts errors to run time. 4. duplicate thorn entries in thornlist, one copy in the CHECKOUT line to have GetComponents check it out and a second (commented out) copy in the bulk so that simfactory can activate it if desired
misc: * Peter will host the call next week
Yours, Roland
On Feb 3, 2014, at 12:42 , Roland Haas roland.haas@physics.gatech.edu wrote:
Present: Frank, Roland, Matt, Erik, Peter
using intel 14 compiler:
- Frank tried and failed with a compiler abort inside of CTGamma
- Erik tried on a machine with both gcc 4.8 and gcc 4.7 installed which
causes the compilation to fail due to improperly chosen include files (see: http://software.intel.com/en-us/forums/topic/489013)
enabled-thorns option:
- triggered by ticket #1532
- the current method is not yet optimal
- several options discussed:
- disable thorns from inside Cactus when they detect hey cannot run,
this needs to be done very early in the build process 2. use two thornlists, one generated automatically from the other. Simfactory or GetComponents may do this. 3. thorns detect at configure time that they cannot compile and turn themselves into No-OP thorns. This shifts errors to run time. 4. duplicate thorn entries in thornlist, one copy in the CHECKOUT line to have GetComponents check it out and a second (commented out) copy in the bulk so that simfactory can activate it if desired
I have implemented option 4, since it is the only one that works with our current infrastructure (Cactus and GetComponents).
-erik
Hi
On Tue, Feb 04, 2014 at 10:39:23PM -0500, Erik Schnetter wrote:
I have implemented option 4, since it is the only one that works with our current infrastructure (Cactus and GetComponents).
This now included Boost, a thorn with a source size alone of half a Gigabyte. I don't think this was really intended, at least not at this point where the complete source is still included in the thorn.
Frank
On Feb 4, 2014, at 23:31 , Frank Loeffler knarf@cct.lsu.edu wrote:
Hi
On Tue, Feb 04, 2014 at 10:39:23PM -0500, Erik Schnetter wrote:
I have implemented option 4, since it is the only one that works with our current infrastructure (Cactus and GetComponents).
This now included Boost, a thorn with a source size alone of half a Gigabyte. I don't think this was really intended, at least not at this point where the complete source is still included in the thorn.
I added download lines for all thorns mentioned in our thorn list.
The compressed source code itself has only about 70 MByte. The remainder is git repository "overhead", i.e. previous Boost versions. Maybe cleaning the git repository would help?
-erik
On 5 Feb 2014, at 06:22, Erik Schnetter schnetter@cct.lsu.edu wrote:
On Feb 4, 2014, at 23:31 , Frank Loeffler knarf@cct.lsu.edu wrote:
Hi
On Tue, Feb 04, 2014 at 10:39:23PM -0500, Erik Schnetter wrote:
I have implemented option 4, since it is the only one that works with our current infrastructure (Cactus and GetComponents).
This now included Boost, a thorn with a source size alone of half a Gigabyte. I don't think this was really intended, at least not at this point where the complete source is still included in the thorn.
I added download lines for all thorns mentioned in our thorn list.
The compressed source code itself has only about 70 MByte. The remainder is git repository "overhead", i.e. previous Boost versions. Maybe cleaning the git repository would help?
Hi,
As far as I remember, Boost is not part of the ET, as it hasn't yet been agreed that it should be added, or in what form (maybe I missed it?). There is a ticket about this, but no conclusion about whether the vote happened or not (https://trac.einsteintoolkit.org/ticket/1410). The minutes of the 8th August meeting do not mention it. I think the discussion started briefly at the end of a later telecon but there was not enough time, and it hasn't been brought up again. Specifically, I think the major objection to Boost is its size, which can be ameliorated by changing the way the repository is structured, or the way that external thorns work (e.g. downloading the library tarball when needed, rather than on checkout). The toolkit is only supposed to contain components which have been agreed on. Since Boost is not yet (as far as I know) part of the toolkit, I don't think it should be downloaded with the toolkit by default. It should be commented out in the thornlist, so that users can add it back in if they want, until we agree on whether it should be included, and in what form.
On Feb 5, 2014, at 5:06 , Ian Hinder ian.hinder@aei.mpg.de wrote:
On 5 Feb 2014, at 06:22, Erik Schnetter schnetter@cct.lsu.edu wrote:
On Feb 4, 2014, at 23:31 , Frank Loeffler knarf@cct.lsu.edu wrote:
Hi
On Tue, Feb 04, 2014 at 10:39:23PM -0500, Erik Schnetter wrote:
I have implemented option 4, since it is the only one that works with our current infrastructure (Cactus and GetComponents).
This now included Boost, a thorn with a source size alone of half a Gigabyte. I don't think this was really intended, at least not at this point where the complete source is still included in the thorn.
I added download lines for all thorns mentioned in our thorn list.
The compressed source code itself has only about 70 MByte. The remainder is git repository "overhead", i.e. previous Boost versions. Maybe cleaning the git repository would help?
Hi,
As far as I remember, Boost is not part of the ET, as it hasn't yet been agreed that it should be added, or in what form (maybe I missed it?). There is a ticket about this, but no conclusion about whether the vote happened or not (https://trac.einsteintoolkit.org/ticket/1410). The minutes of the 8th August meeting do not mention it. I think the discussion started briefly at the end of a later telecon but there was not enough time, and it hasn't been brought up again. Specifically, I think the major objection to Boost is its size, which can be ameliorated by changing the way the repository is structured, or the way that external thorns work (e.g. downloading the library tarball when needed, rather than on checkout). The toolkit is only supposed to contain components which have been agreed on. Since Boost is not yet (as far as I know) part of the toolkit, I don't think it should be downloaded with the toolkit by default. It should be commented out in the thornlist, so that users can add it back in if they want, until we agree on whether it should be included, and in what form.
I don't think we should list thorns in our manifest which are not part of the Einstein Toolkit. I would rather remove it completely. Otherwise, we'll have a set of thorns that are "not officially in the ET", but "mentioned in the manifest for convenience".
Either adding Boost to the ET (after discussion and addressing the issues), or adding it manually to the thorn list if one wants it even though it is not part of the ET are not difficult.
I removed the automated download, and I suggest to remove Boost completely.
There are a few other thorns "in limbo", e.g. MemSpeed. Let's follow the proper procedure here, including a review -- that can't hurt...
-erik
On Feb 5, 2014, at 5:06 , Ian Hinder ian.hinder@aei.mpg.de wrote:
On 5 Feb 2014, at 06:22, Erik Schnetter schnetter@cct.lsu.edu wrote:
On Feb 4, 2014, at 23:31 , Frank Loeffler knarf@cct.lsu.edu wrote:
Hi
On Tue, Feb 04, 2014 at 10:39:23PM -0500, Erik Schnetter wrote:
I have implemented option 4, since it is the only one that works with our current infrastructure (Cactus and GetComponents).
This now included Boost, a thorn with a source size alone of half a Gigabyte. I don't think this was really intended, at least not at this point where the complete source is still included in the thorn.
I added download lines for all thorns mentioned in our thorn list.
The compressed source code itself has only about 70 MByte. The remainder is git repository "overhead", i.e. previous Boost versions. Maybe cleaning the git repository would help?
Hi,
As far as I remember, Boost is not part of the ET, as it hasn't yet been agreed that it should be added, or in what form (maybe I missed it?). There is a ticket about this, but no conclusion about whether the vote happened or not (https://trac.einsteintoolkit.org/ticket/1410). The minutes of the 8th August meeting do not mention it. I think the discussion started briefly at the end of a later telecon but there was not enough time, and it hasn't been brought up again. Specifically, I think the major objection to Boost is its size, which can be ameliorated by changing the way the repository is structured, or the way that external thorns work (e.g. downloading the library tarball when needed, rather than on checkout). The toolkit is only supposed to contain components which have been agreed on. Since Boost is not yet (as far as I know) part of the toolkit, I don't think it should be downloaded with the toolkit by default. It should be commented out in the thornlist, so that users can add it back in if they want, until we agree on whether it should be included, and in what form.
I have copied the Boost thorn into an svn repository, which is significantly smaller. Should we keep this in the ET thorn list, remove it, download it by default...?
-erik
Hi
On Wed, Feb 05, 2014 at 03:30:47PM -0500, Erik Schnetter wrote:
I have copied the Boost thorn into an svn repository, which is significantly smaller. Should we keep this in the ET thorn list, remove it, download it by default...?
We might want to keep it out of the ET thorn list, at least until its status is clear (I mean the discussion whether to actually include it), and maybe even until we made the change to an active download system. However, this move makes its usage a little easier already now (thanks), although it still has a disk foot print 133MB, compared to about 700MB for all other ET thorn checkouts combined, and about 100MB of all other external library thorn checkouts.
Boost is truly a huge beast. Isn't there a possibility to extract the subset we actually need? Maybe using 'bcp' [1] ?
Frank
[1] http://www.boost.org/doc/libs/1_55_0/tools/bcp/doc/html/index.html
On 02/05/2014 09:30 PM, Erik Schnetter wrote:
On Feb 5, 2014, at 5:06 , Ian Hinder ian.hinder@aei.mpg.de wrote:
On 5 Feb 2014, at 06:22, Erik Schnetter schnetter@cct.lsu.edu wrote:
On Feb 4, 2014, at 23:31 , Frank Loeffler knarf@cct.lsu.edu wrote:
Hi
On Tue, Feb 04, 2014 at 10:39:23PM -0500, Erik Schnetter wrote:
I have implemented option 4, since it is the only one that works with our current infrastructure (Cactus and GetComponents).
This now included Boost, a thorn with a source size alone of half a Gigabyte. I don't think this was really intended, at least not at this point where the complete source is still included in the thorn.
I added download lines for all thorns mentioned in our thorn list.
The compressed source code itself has only about 70 MByte. The remainder is git repository "overhead", i.e. previous Boost versions. Maybe cleaning the git repository would help?
Hi,
As far as I remember, Boost is not part of the ET, as it hasn't yet been agreed that it should be added, or in what form (maybe I missed it?). There is a ticket about this, but no conclusion about whether the vote happened or not (https://trac.einsteintoolkit.org/ticket/1410). The minutes of the 8th August meeting do not mention it. I think the discussion started briefly at the end of a later telecon but there was not enough time, and it hasn't been brought up again. Specifically, I think the major objection to Boost is its size, which can be ameliorated by changing the way the repository is structured, or the way that external thorns work (e.g. downloading the library tarball when needed, rather than on checkout). The toolkit is only supposed to contain components which have been agreed on. Since Boost is not yet (as far as I know) part of the toolkit, I don't think it should be downloaded with the toolkit by default. It should be commented out in the thornlist, so that use
rs can add it back in if they want, until we agree on whether it should be included, and in what form.
I have copied the Boost thorn into an svn repository, which is significantly smaller. Should we keep this in the ET thorn list, remove it, download it by default...?
-erik
Hi,
have you looked into the possibility of using git-annex for large binary files? It is a tool that acts on top of a git repository and it commits only symbolic links to the data you choose into the git repository. A checkout then would be quick and you could choose to bring in from a central repository only the data you want if or when you need it. Please take a look into the following link for more info:
https://git-annex.branchable.com/
specially the walk through link:
https://git-annex.branchable.com/walkthrough/
It may be helpful when dealing with external libraries tar balls.
Cheers, Bruno.
On Wed, Feb 05, 2014 at 11:26:32PM +0100, Bruno Coutinho Mundim wrote:
have you looked into the possibility of using git-annex for large binary files?
While git-annex sounds interesting, whatever tool we use has to be installed on the system before we even start to compile anything. We would need to make GetComponents depend on git-annex being installed on each system it is used on. Since git-annex is not part of standard git, and not such a wide-spread tool like git, this would speak against it.
Frank
On Feb 3, 2014, at 12:42 , Roland Haas roland.haas@physics.gatech.edu wrote:
Present: Frank, Roland, Matt, Erik, Peter
using intel 14 compiler:
- Frank tried and failed with a compiler abort inside of CTGamma
- Erik tried on a machine with both gcc 4.8 and gcc 4.7 installed which
causes the compilation to fail due to improperly chosen include files (see: http://software.intel.com/en-us/forums/topic/489013)
I uninstalled gcc 4.7, and added the line
SYS_INC_DIRS = /usr/include/x86_64-linux-gnu/c++/4.8
to the option list. This made Intel 14 work for me.
-erik
users@lists.einsteintoolkit.org
-
Bruno Coutinho Mundim -
Erik Schnetter -
Frank Loeffler -
Frank Löffler
-
Ian Hinder -
Roland Haas