[Users] Meeting Minutes

[Ian Hinder]

On 2 Dec 2013, at 20:38, Frank Loeffler <knarf at cct.lsu.edu> wrote:

> On Mon, Dec 02, 2013 at 11:20:20AM -0800, Roland Haas wrote:
>> * suggestion to not include that tarball in ExternalLibraries, instead
>> download in configure.sh script of thorn only if needed. Not yet fully
>> agreed on whether we want this or not.

I have thought about this a fair bit, and Barry and I discussed it briefly as well.  Here is my current proposal:

	• The tarball would not be included in the repository
	• The configuration.ccl file of the thorn would have a field for the URL of a tarball to download
	• At configuration time, the thorn's configure script (or something factored into a Cactus script) would download the tarball using the URL if it is necessary to build the library on that machine.
	• The tarball would be cached in Cactus/librarycache or similar
	• A script or makefile target would be provided to "pre-cache" the tarball of one or more thorns

A checkout of the ET now would not download any external library tarballs.  If, on a given machine, a library is not installed and needs to be built, it would be downloaded when needed. If you want to make sure you have everything needed to build, for example if you are about to catch a plane, you could run something like "make get-libraries" and all library tarballs would be downloaded into Cactus/librarycache.  [It would be possible to do this using simfactory, which might be able to determine via its machine database which machines need which tarballs downloaded].  You could then choose to sync this to the remote machine, or not.  On a remote machine, you could run get-libraries on the head node which presumably has an internet connection so that the library tarballs are available to the build process which might, as Frank points out, not have internet access.

The vast majority of users would notice little difference; the checkout would be faster, and less disk space would be used by their Cactus trees.  Edge-case users who have to compile on machines without internet connections would have a simple command to run on the head node of the machine which would restore the same functionality that we currently have.

> This can sometimes be difficult to make work. In some cases one has to
> build on a compute node (e.g. when some libraries are only installed
> there, e.g. GPU-related libraries - this is currently the case for
> instance on supermike). However, compute nodes might not be able to make
> connections directly to the Internet, or there might be policies against
> it.

Right; it can be done on the head node instead, and used by the build process on the compute nodes.

> I would rather like a solution that downloads the tarball together with
> the thorn.
> 
> This is primarily a problem for large libraries though, especially when
> a corresponding system library exists and is used. It doesn't only
> increase the size of a checkout, it also increases the size of the
> executable (when using Formaline). We could exclude external libraries
> from Formaline (at least the dist subdirectory) - system libraries are
> not included as well after all.
> This would leave us with the problem of the large checkout. I don't
> think the download is so much of a problem, but the disk space might be.
> As long as it is a tarball it should at least not be a huge inode
> problem, and can easily be compressed.


-- 
Ian Hinder
http://numrel.aei.mpg.de/people/hinder

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20131202/bd0a5c3b/attachment.bin