Present: Vassili, Frank, Erik, Ian, Roland
* we decided before the release to switch our repositories to git, once LSU has finished software upgrades and Frank has time
* would like to (also) run tests for "optional" ET components that are currently commented out and not tested automatically
Removal of thorns from main ET thorn list/deprecation:
* we had a lively discussion as to how to proceed * uncontroversial: move ADMConstraints into EinsteinArchive and replace with ML_ADMConstraint which provides the same functionality. Reason: ADMConstraints uses old Tmunu interface (optionally) which is deprecated and will be removed. ADMConstraints also uses ADMMacros (is 4th order only etc) which are hard to maintain (we also found a bug in 2010: r107 "Undefine the guts instead of the declare") * uncontroversial: remove old Tmunu interface. Reason: eat up ~30% of total CPU time since it contains a CCTK_IsThornActive in the innermost loop * uncontroversial: retire ADMCoupling since it is only used for the deprecated Tmunu interface, FRIEND mechanism in Cactus is also no longer advertised as good usage. Requires fixes in Exact. * ADMMacros is used by AHFinder. AHFinder seems to work (passes tests) though no one (of the callers) has used it for a long time. Difficult to motivate to why we retire it. * DistortedBHIVP, IDAxiBrillBH, IDAxiOddBrillBH, RoatingDBHIVP: mark as archived using comment. Remove from manifest (comment out) if tests fail. There are objections to moving the repository.
New thorns:
* CoreCollapseControl: needs example and/or test * CartesianCoordinates: rework into "MultiBlockBase/JacobianBase" and have Llama thorns inherit from it. Keep providing trivial coordinates (same as ADMBase providing Minkowski spacetime) * ZelmaniQuadWaveExtract: needs docs/test cases/examples. Otherwise this is something we'd like. * PAPI: want documentation, include * MemSpeed: want documentation, test case, include * dylib: include if compiles, WARN loudly if it cannot run at runtime * F5: want documentation, include * WaveToyCUDA: keep commented out, assuming testsuites and docs * Boost: objections to size of repo. ran out of time to reach conclusion
Related issues:
* this idea of having multiple thornlists, one containing a subset and one containing everything was tried in the past and found not to work (since they are hard to keep in sync). There needs to be a master thornlist. * want mechanism to mark tests as "know to fail". Most likely needs to be done in simfactory's files since failures are machine specific * suggestion to not include that tarball in ExternalLibraries, instead download in configure.sh script of thorn only if needed. Not yet fully agreed on whether we want this or not. * no agreed upon way to "retire" a thorn right now, different opinions on how to do this.
Yours, Roland
On Mon, Dec 02, 2013 at 11:20:20AM -0800, Roland Haas wrote:
- suggestion to not include that tarball in ExternalLibraries, instead
download in configure.sh script of thorn only if needed. Not yet fully agreed on whether we want this or not.
This can sometimes be difficult to make work. In some cases one has to build on a compute node (e.g. when some libraries are only installed there, e.g. GPU-related libraries - this is currently the case for instance on supermike). However, compute nodes might not be able to make connections directly to the Internet, or there might be policies against it.
I would rather like a solution that downloads the tarball together with the thorn.
This is primarily a problem for large libraries though, especially when a corresponding system library exists and is used. It doesn't only increase the size of a checkout, it also increases the size of the executable (when using Formaline). We could exclude external libraries from Formaline (at least the dist subdirectory) - system libraries are not included as well after all. This would leave us with the problem of the large checkout. I don't think the download is so much of a problem, but the disk space might be. As long as it is a tarball it should at least not be a huge inode problem, and can easily be compressed.
Frank
On 2 Dec 2013, at 20:38, Frank Loeffler knarf@cct.lsu.edu wrote:
On Mon, Dec 02, 2013 at 11:20:20AM -0800, Roland Haas wrote:
- suggestion to not include that tarball in ExternalLibraries, instead
download in configure.sh script of thorn only if needed. Not yet fully agreed on whether we want this or not.
I have thought about this a fair bit, and Barry and I discussed it briefly as well. Here is my current proposal:
• The tarball would not be included in the repository • The configuration.ccl file of the thorn would have a field for the URL of a tarball to download • At configuration time, the thorn's configure script (or something factored into a Cactus script) would download the tarball using the URL if it is necessary to build the library on that machine. • The tarball would be cached in Cactus/librarycache or similar • A script or makefile target would be provided to "pre-cache" the tarball of one or more thorns
A checkout of the ET now would not download any external library tarballs. If, on a given machine, a library is not installed and needs to be built, it would be downloaded when needed. If you want to make sure you have everything needed to build, for example if you are about to catch a plane, you could run something like "make get-libraries" and all library tarballs would be downloaded into Cactus/librarycache. [It would be possible to do this using simfactory, which might be able to determine via its machine database which machines need which tarballs downloaded]. You could then choose to sync this to the remote machine, or not. On a remote machine, you could run get-libraries on the head node which presumably has an internet connection so that the library tarballs are available to the build process which might, as Frank points out, not have internet access.
The vast majority of users would notice little difference; the checkout would be faster, and less disk space would be used by their Cactus trees. Edge-case users who have to compile on machines without internet connections would have a simple command to run on the head node of the machine which would restore the same functionality that we currently have.
This can sometimes be difficult to make work. In some cases one has to build on a compute node (e.g. when some libraries are only installed there, e.g. GPU-related libraries - this is currently the case for instance on supermike). However, compute nodes might not be able to make connections directly to the Internet, or there might be policies against it.
Right; it can be done on the head node instead, and used by the build process on the compute nodes.
I would rather like a solution that downloads the tarball together with the thorn.
This is primarily a problem for large libraries though, especially when a corresponding system library exists and is used. It doesn't only increase the size of a checkout, it also increases the size of the executable (when using Formaline). We could exclude external libraries from Formaline (at least the dist subdirectory) - system libraries are not included as well after all. This would leave us with the problem of the large checkout. I don't think the download is so much of a problem, but the disk space might be. As long as it is a tarball it should at least not be a huge inode problem, and can easily be compressed.
On Mon, Dec 2, 2013 at 3:16 PM, Ian Hinder ian.hinder@aei.mpg.de wrote:
This can sometimes be difficult to make work. In some cases one has to build on a compute node (e.g. when some libraries are only installed there, e.g. GPU-related libraries - this is currently the case for instance on supermike). However, compute nodes might not be able to make connections directly to the Internet, or there might be policies against it.
Right; it can be done on the head node instead, and used by the build process on the compute nodes.
For machines that have this issue, SimFactory should be able to know about it and ensure the necessary library tarballs are available at job submission time.
On Mon, Dec 02, 2013 at 03:22:17PM -0500, Barry Wardell wrote:
For machines that have this issue, SimFactory should be able to know about it and ensure the necessary library tarballs are available at job submission time.
When you build on a compute node you don't necessarily use simfactory to get the interactive job. However, the make-target Ian described should be sufficient for these cases.
Frank
On Mon, Dec 02, 2013 at 09:16:01PM +0100, Ian Hinder wrote:
• The configuration.ccl file of the thorn would have a field for the URL of a tarball to download
The name of the tarball which is to be downloaded probably should contain something like a version of the library. This means that if a thorn gets updated (because of an updated version for example), old checkouts of the thorn would still try to access the old version - which means we would need to keep it around forever. Do we want this? This is not a rhetorical question, the answer might as well be "yes". In fact, that would probably be the most workable solution. We might even stuff this tarball into a svn repository and download it directly from there, without a checkout. git wouldn't be usable for that.
Frank
On Mon, Dec 2, 2013 at 3:24 PM, Frank Loeffler knarf@cct.lsu.edu wrote:
On Mon, Dec 02, 2013 at 09:16:01PM +0100, Ian Hinder wrote:
• The configuration.ccl file of the thorn would have a field forthe URL of a tarball to download
The name of the tarball which is to be downloaded probably should contain something like a version of the library. This means that if a thorn gets updated (because of an updated version for example), old checkouts of the thorn would still try to access the old version - which means we would need to keep it around forever. Do we want this? This is not a rhetorical question, the answer might as well be "yes".
In my opinion, the answer is "yes" since otherwise it would be impossible to guarantee reproducibility.
On Mon, Dec 02, 2013 at 03:30:28PM -0500, Barry Wardell wrote:
In my opinion, the answer is "yes" since otherwise it would be impossible to guarantee reproducibility.
I agree - except in case you use the system libraries you don't have the same level of reproducibility anyway.
Frank
On 12/02/2013 01:20 PM, Roland Haas wrote:
Present: Vassili, Frank, Erik, Ian, Roland
and Steve
- we decided before the release to switch our repositories to git, once
LSU has finished software upgrades and Frank has time
- would like to (also) run tests for "optional" ET components that are
currently commented out and not tested automatically
Removal of thorns from main ET thorn list/deprecation:
- we had a lively discussion as to how to proceed
- uncontroversial: move ADMConstraints into EinsteinArchive and replace
with ML_ADMConstraint which provides the same functionality. Reason: ADMConstraints uses old Tmunu interface (optionally) which is deprecated and will be removed. ADMConstraints also uses ADMMacros (is 4th order only etc) which are hard to maintain (we also found a bug in 2010: r107 "Undefine the guts instead of the declare")
- uncontroversial: remove old Tmunu interface. Reason: eat up ~30% of
total CPU time since it contains a CCTK_IsThornActive in the innermost loop
- uncontroversial: retire ADMCoupling since it is only used for the
deprecated Tmunu interface, FRIEND mechanism in Cactus is also no longer advertised as good usage. Requires fixes in Exact.
- ADMMacros is used by AHFinder. AHFinder seems to work (passes tests)
though no one (of the callers) has used it for a long time. Difficult to motivate to why we retire it.
- DistortedBHIVP, IDAxiBrillBH, IDAxiOddBrillBH, RoatingDBHIVP: mark as
archived using comment. Remove from manifest (comment out) if tests fail. There are objections to moving the repository.
New thorns:
- CoreCollapseControl: needs example and/or test
- CartesianCoordinates: rework into "MultiBlockBase/JacobianBase" and
have Llama thorns inherit from it. Keep providing trivial coordinates (same as ADMBase providing Minkowski spacetime)
- ZelmaniQuadWaveExtract: needs docs/test cases/examples. Otherwise this
is something we'd like.
- PAPI: want documentation, include
- MemSpeed: want documentation, test case, include
- dylib: include if compiles, WARN loudly if it cannot run at runtime
- F5: want documentation, include
- WaveToyCUDA: keep commented out, assuming testsuites and docs
- Boost: objections to size of repo. ran out of time to reach conclusion
Related issues:
- this idea of having multiple thornlists, one containing a subset and
one containing everything was tried in the past and found not to work (since they are hard to keep in sync). There needs to be a master thornlist.
- want mechanism to mark tests as "know to fail". Most likely needs to
be done in simfactory's files since failures are machine specific
- suggestion to not include that tarball in ExternalLibraries, instead
download in configure.sh script of thorn only if needed. Not yet fully agreed on whether we want this or not.
- no agreed upon way to "retire" a thorn right now, different opinions
on how to do this.
Yours, Roland
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
users@lists.einsteintoolkit.org
-
Barry Wardell -
Frank Loeffler -
Ian Hinder -
Roland Haas -
Steven R. Brandt