Trac April 2016

trac@lists.einsteintoolkit.org

1 participants
107 discussions

[Einstein Toolkit] #1812: GetComponents needs to come with (a selected few) trusted certificates
by Einstein Toolkit 07 May '16

07 May '16

#1812: GetComponents needs to come with (a selected few) trusted certificates ---------------------------+------------------------------------------------ Reporter: knarf | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: GetComponents | Version: development version Keywords: | ---------------------------+------------------------------------------------ Most encrypted connections use some form of certificates. We use both ssl for subversion and git. The problem with certificates is that the certificate of the root CA has to be trusted by a client to be able to form a trusted connection. Otherwise users get warnings, and either cannot connect at all, or have to manually approve. We don't want both happening with GetComponents. Root CA certificates have to be changed from time to time. They have a limited life time. In addition, there is a big movement away from SHA-1 certificates right now, as this has been found weak. The problem with that is that an updated root CA might not have it made to all (or at least most) clients at the time a user might have to trust it. Usually, root CAs are created, but not actually used for some time for that reason. Security problems like SHA-1 interfere with that. All this is nothing we can change. So, in the light of root CAs possibly faster changing than clients update (and especially supercomputers update slowly in particular), we have to have a mechanism to make GetComponents still work 'out of the box' with these - no matter what the protocol is that actually uses it. Right now, we only seem to have problems with svn, but in the future we might have the same problem with git (bitbuckets root CA is still using SHA-1). For Subversion I propose the following. We ship 'too new' certificates with GetComponents (in-source, to still have one file). We then write this to a temporary file during checkout, and use the following option to svn to get it to accept it: {{{ --config-option servers:global:ssl-authority-files=<FILE> }}} This mechanism was tested manually on one of the problematic machines (stampede), with one of the problematic certs (InCommon). This only works for svn version 1.6 and newer. Most installations should have that. In case we encounter an older one we could disable trusted cert checking for problematic cases using --trust-server-cert. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1812> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 6

[Einstein Toolkit] #1877: TimerReport change requires C++11, but Cactus does not
by Einstein Toolkit 02 May '16

02 May '16

#1877: TimerReport change requires C++11, but Cactus does not --------------------+------------------------------------------------------- Reporter: hinder | Owner: Type: defect | Status: new Priority: major | Milestone: Component: Cactus | Version: development version Keywords: | --------------------+------------------------------------------------------- The commit https://bitbucket.org/cactuscode/cactusutils/commits/5c812eb236f4ff033fc1c2… uses C++11 features which are not enabled by Cactus autoconf. Erik warned in the ET call yesterday that this might be a problem. The error you get is Checking status of thorn TimerReport In file included from /usr/include/c++/4.8/array:35:0, from /home/jenkins/workspace/KrancTutorial/Cactus/arrangements/CactusUtils/TimerReport/src/Output.cc:16: /usr/include/c++/4.8/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options. #error This file requires compiler and library support for the This is when compiling without an optionlist on Ubuntu 14.04. * If we want to require C++11 for Cactus, then we should configure for it (i.e. add the -std=gnu++11 option when configuring, as is done in several simfactory optionlists). * My feeling is that C++11 compiler support might not be diffuse enough or mature enough to require for Cactus, which prides itself on portability, but I'm not up-to-date on the current C++ landscape. The above error message, on a version of Ubuntu which has been LTS up to a week ago, suggests that the GCC authors were also not sufficiently confident in the implementation. The [[https://gcc.gnu.org/projects/cxx- status.html#cxx11|GCC C++11 page]] doesn't seem to list recent GCC versions, but the latest version listed (4.8) still describes the support as "experimental". * Maybe the C++11 features being used in TimerReport are not critical, and can be easily worked around? -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1877> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #1879: Remove the need for McLachlan helper thorns
by Einstein Toolkit 28 Apr '16

28 Apr '16

#1879: Remove the need for McLachlan helper thorns -----------------------------------+---------------------------------------- Reporter: eschnett | Owner: Type: enhancement | Status: new Priority: optional | Milestone: Component: EinsteinToolkit thorn | Version: development version Keywords: | -----------------------------------+---------------------------------------- McLachlan generates helper thorns. These should be converted to using the new "merge" mechanism. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1879> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 0

[Einstein Toolkit] #1878: hdf5 deflate fragments memory
by Einstein Toolkit 27 Apr '16

27 Apr '16

#1878: hdf5 deflate fragments memory -------------------------+-------------------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: Other | Version: development version Keywords: HDF5 | -------------------------+-------------------------------------------------- In ticket #1874, a problem with memory management of the HDF5 zlib deflate filter was mentioned. This ticket here was opened to collect information about this problem, with the aim of eventually providing a patch to HDF5, submitting a ticket to the HDF5 developers, and possibly implementing that change as patch just for the ET. The problem is that HDF5 allocates while deflating an output buffer (for the uncompressed data) of the size of the compressed data, and then makes it larger (by a factor of two - another problem), if needed (almost always). This happens in H5Z_filter_deflate() in H5Zdeflate.c . One way to solve this would be to use the z_strm.total_in value, which holds the size of the uncompressed data, saved by the compressor. However, I am currently unsure whether this is a value we can always rely on. The format specification mentions it as required field, while the zlib documentation talks about it as "may have been saved by the compressor". I didn't look at any actual data. In any case, if this value is present, we can use it to allocate the output buffer of the correct size. If it isn't present, the best we could do is at least not increase the size by a factor of two, which is known to cause memory fragmentation. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1878> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 2

[Einstein Toolkit] #1689: Cactus should exit with a nonzero exit code if an error occurs
by Einstein Toolkit 27 Apr '16

27 Apr '16

#1689: Cactus should exit with a nonzero exit code if an error occurs --------------------+------------------------------------------------------- Reporter: hinder | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: Cactus | Version: development version Keywords: | --------------------+------------------------------------------------------- In WarnLevel.c, in CCTK_VWarn (called by CCTK_Warn), it says if (level <= error_level) { CCTK_Abort (NULL, 0); } The second argument to CCTK_Abort is the exit code of the process. So if there is an "error" warning, the process exits with 0 exit code; i.e. success! This happens in several places in this file. The user guide does not say anything about the exit code of Cactus. I think that if Cactus has a level-0 warning, i.e. an error, then it should exit with a non-zero exit code, and Erik agrees. See http://lists.einsteintoolkit.org/pipermail/users/2014-November/003878.html -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1689> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 4

[Einstein Toolkit] #1874: Activate HDF5 shuffle filter
by Einstein Toolkit 27 Apr '16

27 Apr '16

#1874: Activate HDF5 shuffle filter -----------------------------------+---------------------------------------- Reporter: eschnett | Owner: Type: enhancement | Status: new Priority: unset | Milestone: Component: EinsteinToolkit thorn | Version: development version Keywords: | -----------------------------------+---------------------------------------- The HDF5 shuffle filter increases the efficiency of the HDF5 deflate filter. See <https://bitbucket.org/cactuscode/cactuspughio/pull-requests/2 /iohdf5util-activate-shuffle-filter/diff> and <https://bitbucket.org/eschnett/carpet/pull-requests/11/eschnett- hdf5-shuffle/diff>. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1874> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 6

[Einstein Toolkit] #1876: revision 132 of HDF5 fails to build
by Einstein Toolkit 25 Apr '16

25 Apr '16

#1876: revision 132 of HDF5 fails to build -----------------------------------+---------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: blocker | Milestone: Component: EinsteinToolkit thorn | Version: development version Keywords: HDF5 | -----------------------------------+---------------------------------------- revision 132 of HDF5 fails to build in the Jenkins test servers: https://build.barrywardell.net/job/EinsteinToolkit/735/changes A first glance at the output show that in configure.ccl it refers to detect.pl which does not exist (instead of detect.sh). This is a blocker since an ET thorn fails to compile. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1876> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #1864: Cactus build fails on file systems that do not update timestamps all the time
by Einstein Toolkit 22 Apr '16

22 Apr '16

#1864: Cactus build fails on file systems that do not update timestamps all the time --------------------+------------------------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: major | Milestone: Component: Cactus | Version: development version Keywords: | --------------------+------------------------------------------------------- Ian and I recently ran into an issue caused by a particular file system (NFS v4 and also BeeGFS) not updating the file modification time when doing this: {{{ : >make.checked }}} which is what the build system uses (since [https://bitbucket.org/cactuscode/cactus/commits/0779c17697d3b4a254065c10836… 0779c17697d3b4a254065c10836770e355071f41] "Cactus: Replace "echo" by ":" in makefile" Thu Nov 27 16:35:13 2014 -0500) to update the marker files once a directory is finished building. While such a behaviour is not POSIX compliant (https://bugzilla.kernel.org/show_bug.cgi?id=6127), we'd still want to work around it. The simplest solution seems to me to revert [https://bitbucket.org/cactuscode/cactus/commits/0779c17697d3b4a254065c10836… 0779c17697d3b4a254065c10836770e355071f41] and use {{{ echo "" >make.checked }}} again. Erik: since you made the change, would you see any downside to reverting it? While investigating this Ian also found that some file systems only offer 1 second granularity in their timestamps (eg ext3 but also possibly XFS and NFS) which can negatively affect make if a rule takes less than a second to complete. See https://savannah.gnu.org/bugs/?40056#comment0 and https://www.gnu.org/software/autoconf/manual/autoconf-2.61/html_node /Timestamps-and-Make.html . The most conservative approach would be to (arrange for) {{{sleep 1}}} to execute after each make recipe. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1864> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 8

[Einstein Toolkit] #1837: LocalInterp2 symmetry test fails on intel compiler machine
by Einstein Toolkit 19 Apr '16

19 Apr '16

#1837: LocalInterp2 symmetry test fails on intel compiler machine -----------------------------------+---------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Keywords: LocalInterp2 | -----------------------------------+---------------------------------------- The intel compiler by default aggressively optimizes expressions, which causes failures in the delicate cancellations required for LocalInterp2's symmetry test. The attached patch changes compiler options for the affected file only. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1837> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 4

[Einstein Toolkit] #702: Tests should use IO::out_fileinfo = "none"
by Einstein Toolkit 18 Apr '16

18 Apr '16

#702: Tests should use IO::out_fileinfo = "none" -------------------------+-------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: Other | Version: Keywords: | -------------------------+-------------------------------------------------- The Cactus User Guide (http://einsteintoolkit.org/documentation/UsersGuide/UsersGuidech9.html#x13-…) recommends that test output files should always be the same, and hence use IO::out_fileinfo = "none". I would like to implement this for the tests in the ET, as it makes comparing test output using standard (non-Cactus testsuite mechanism) diff tools possible. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/702> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 10

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Trac April 2016