Trac August 2015

trac@lists.einsteintoolkit.org

1 participants
112 discussions

[Einstein Toolkit] #697: Repository and thorn names cannot be different
by Einstein Toolkit 10 Nov '15

10 Nov '15

#697: Repository and thorn names cannot be different ---------------------------+------------------------------------------------ Reporter: eschnett | Owner: eric9 Type: defect | Status: new Priority: critical | Milestone: Component: GetComponents | Version: Keywords: | ---------------------------+------------------------------------------------ I am accessing some thorns stored in git repositories at bitbucket.org, which insists that all repository names are all lower case. I don't know how to specify this in GetComponents. When I say !TARGET = $ARR !TYPE = git !URL = git@bitbucket.org:user/thorn.git !AUTH_URL = git@bitbucket.org:user/thorn.git !CHECKOUT = Arrangement/Thorn then GetComponents does not create a symbolic link from Thorn to ../../repos/thorn, but to ../../repos/thorn/Arrangement/Thorn instead. How do I avoid this? -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/697> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #241: Thorns should be able to specify implementation versions, and other thorns should be able to depend on those
by Einstein Toolkit 26 Oct '15

26 Oct '15

#241: Thorns should be able to specify implementation versions, and other thorns should be able to depend on those -------------------------+-------------------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: Cactus | Version: Keywords: | -------------------------+-------------------------------------------------- Cactus thorns implement 'implementations', and other thorns can then depend on particular implementations being present. However, quite often those are updated, and incompatibilities occur. Usually this is resolved by the developer in both thorns - the implementing and the using thorn. However, a user might just update one of them, and Cactus would not catch this error. Thus, I suggest to think about, and implement how to: - specify implementation versions e.g. IMPLEMENTS: Cactus (4.0) The version string within () would have be sorted in an intelligent way to allow comparisons [1]. - specify dependencies on particular versions of other implementations In particular I suggest the following dependencies: depends, and conflicts, both with comparisons < <= == >= > != e.g. DEPENDS: Cactus (>= 4.0) DEPENDS: BadImplementation (!=3.14) <-- this introduces a dependency CONFLICTS: BadImplementation (==3.14) <-- this doesn't introduce a depencency [1] First the initial part of each string consisting entirely of non-digit characters is determined. These two parts (one of which may be empty) are compared lexically. If a difference is found it is returned. The lexical comparison is a comparison of ASCII values modified so that all the letters sort earlier than all the non-letters. Then the initial part of the remainder of each string which consists entirely of digit characters is determined. The numerical values of these two parts are compared, and any difference found is returned as the result of the comparison. For these purposes an empty string (which can only occur at the end of one or both version strings being compared) counts as zero. These two steps (comparing and removing initial non-digit strings and initial digit strings) are repeated until a difference is found or both strings are exhausted. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/241> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 27

[Einstein Toolkit] #1796: Include IllinoisGRMHD into the Toolkit, as an Arrangement
by Einstein Toolkit 20 Oct '15

20 Oct '15

#1796: Include IllinoisGRMHD into the Toolkit, as an Arrangement -----------------------------------+---------------------------------------- Reporter: zachetie@… | Owner: Zachariah Etienne Type: enhancement | Status: new Priority: major | Milestone: ET_2015_11 Component: EinsteinToolkit thorn | Version: development version Keywords: GRMHD IllinoisGRMHD | -----------------------------------+---------------------------------------- IllinoisGRMHD, an open-source, compact rewrite of the Illinois NR group's GRMHD code, has been graciously hosted by the ET community in its BitBucket repo for more than a year, under the EinsteinEvolve arrangement. This ticket requests official inclusion of an updated IllinoisGRMHD and closely-related thorns into the ET, as an arrangement. IllinoisGRMHD brings a number of key new features to the Toolkit, including: 1) a staggered, vector-potential-based, AMR-compatible GRMHD scheme that automatically preserves divergenceless B-fields without the need for specialized interpolation schemes or divergence cleaning. 2) excision-less, robust modeling of GRMHD flows into black hole interiors 3) a highly-robust conservative-to-primitive solver, which checks the physicality of the conservative variables *prior* to inversion, and modifies them minimally to restore physicality. The proposed arrangement will include: 1) IllinoisGRMHD: Core GRMHD evolution routines, some significant bugfixes since the original version currently in the ET's bitbucket repo; about 3,400 lines of code (cloc) 2) ID_converter_ILGRMHD: Converts HydroBase variables into variables IllinoisGRMHD can read (e.g., IllinoisGRMHD uses a velocity definition consistent with the magnetic induction equation, not the Valencia formulation); about 209 lines of code (cloc) 3) convert_to_HydroBase: Does the reverse of ID_converter_ILGRMHD, needed for compatibility with HydroBase-based analysis thorns; 126 lines of code. All codes are written in C99 and are fully OpenMP-ified. You can read more about IllinoisGRMHD in its code announcement paper (CQG in press): http://arxiv.org/abs/1501.07276 Additionally, a Guide to Getting Started with IllinoisGRMHD has also been written and is available at the IllinoisGRMHD webpage: http://math.wvu.edu/~zetienne/ILGRMHD/ You may now download all three of these thorns (licensed GNU GPL v2 or higher) from: math.wvu.edu/~zetienne/IllinoisGRMHD_arrangement_July_20_2015.tar.gz -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1796> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 22

[Einstein Toolkit] #1794: Backport MPI detect.pl changes from development to release version
by Einstein Toolkit 19 Oct '15

19 Oct '15

#1794: Backport MPI detect.pl changes from development to release version ----------------------+----------------------------------------------------- Reporter: hinder | Owner: Type: defect | Status: new Priority: major | Milestone: Component: Other | Version: development version Keywords: backport | ----------------------+----------------------------------------------------- According to the ET mailing list discussion at http://lists.einsteintoolkit.org/pipermail/users/2015-July/004312.html, the ET does not build on Mac OS with MacPorts in the released version, probably due to changes in MacPorts since the release. The fix was committed to the MPI thorn detect.pl script and needs to be backported to the release. Comer Duncan confirms that this fixes the problem. The required commit seems to be http://git.barrywardell.net/?p=arrangements/ExternalLibraries/MPI.git;a=blo…. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1794> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 4

[Einstein Toolkit] #1802: move AEIThorns into git repositories
by Einstein Toolkit 08 Oct '15

08 Oct '15

#1802: move AEIThorns into git repositories -----------------------------------------------------------------------------+ Reporter: rhaas | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Keywords: ADMMass AEILocalInterp PunctureTracker SystemStatistics Trigger | -----------------------------------------------------------------------------+ Currently the ET contains the following AEI thorns: * AEIThorns/ADMMass * AEIThorns/AEILocalInterp * AEIThorns/PunctureTracker * AEIThorns/SystemStatistics * AEIThorns/Trigger since the AEI would like to shut down the svn server (or at least change it to read only mode) it would be good to move these thorns into: EinsteinAnalysis:: ADMMass PunctureTracker CactusUtils:: SystemStatistics Trigger (need change of license to LGPL) Numerical:: AEILocalInterp (license must stay GPL) For the ET thorns Barry kindly did the conversion using a set of scripts of his. If you were to make the scripts accessible (or give the whole thing a try yourself), then this could be started as soon as the authors (Ian and Frank are main authors if SystemStatistics and Trigger, Erik, Barry and I contributed, I am fine with a license change) agree to the license change where required. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1802> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 18

[Einstein Toolkit] #1803: Move LSUThorns into git repositories at Bitbucket
by Einstein Toolkit 08 Oct '15

08 Oct '15

#1803: Move LSUThorns into git repositories at Bitbucket -------------------------+-------------------------------------------------- Reporter: eschnett | Owner: Type: enhancement | Status: new Priority: unset | Milestone: Component: Other | Version: development version Keywords: | -------------------------+-------------------------------------------------- -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1803> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 10

[Einstein Toolkit] #1033: support a combined "map" file in CarpetIOHDF5
by Einstein Toolkit 07 Oct '15

07 Oct '15

#1033: support a combined "map" file in CarpetIOHDF5 --------------------------+------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: enhancement | Status: new Priority: minor | Milestone: Component: Carpet | Version: Keywords: CarpetIOHDF5 | --------------------------+------------------------------------------------- recovering on different number of processes than was used to write a checkpoint is painfully slow. Part of the reason seems to be that each process essentially has to read all files to find out where each piece of data it requires is located. The attached patch (not to be included in the main code due to bad file formats and coding) enables CarpetIOHDF5 to read all the information stored in the union of index files to from a single file. This means (together with the other patches proposed today) that CarpetIOHDF5 only ever opens those HDF5 files that are required to restore the simulation on a given process. It significantly (factor > 4 where I don't quite know how fast since the unpatched version ran out of walltime) speeds up recovery with many more processors than wrote the files. It also adds an optimization for CCTK_VarIndex calls inside CarpetIOHDF5 (which happens for every dataset in the file). This is intended only as a proof of what might speed up recovery. A proper implementation would need a more sensible file format. Two option seem possible: 1) extend the index file format by a "filename" or "filenum" attribute to each dataset and use a concatenation of all index files as the map file 2) define a custom hdf5 data type corresponding to the information in a single patch_t, which would have mostly integer field plus two variable length / enumerated ASCII fields (for the patch name, variable name) -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1033> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #1737: upate CarpetHDF5 reader in VisIt
by Einstein Toolkit 15 Sep '15

15 Sep '15

#1737: upate CarpetHDF5 reader in VisIt ------------------------------+--------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: Other | Version: development version Keywords: VisIt CarpetHDF5 | ------------------------------+--------------------------------------------- Since the CarpetHDF5 reader (https://svn.cactuscode.org/VizTools/CarpetHDF5) was included in VisIt's main source code repo, we have not attempted to update that copy to the most current version. The coding style in that branch matches the official code but the patches do not apply on top of the official code due to ordering issues as well as some minor new changes to code formatting in the current (2.8.0) VisIt codebase. Since then, I have collected a number of bugfixes/improvements that are collected in various branches at https://bitbucket.org/rhaas80/carpethdf5 . It would be good to eventually try and get the for_VisIt branch (https://bitbucket.org/rhaas80/carpethdf5/branch/for_VisIt) included in the main source code repo again. It contains some bug-fixes wrt how file metadata is cached, as well as number of fixes to reduce memory footprint, number of open files (required to work on large datasets on machines that limit the total number of open files), some improvements to error reporting and robustness when dealing with partially corrupted filesets as well as changes that allow a user to combine multiple HDF5 files into a single "virtual" VisIt database using .visit files. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1737> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #1045: URL field should be optional
by Einstein Toolkit 01 Sep '15

01 Sep '15

#1045: URL field should be optional ---------------------------+------------------------------------------------ Reporter: hinder | Owner: eric9 Type: defect | Status: new Priority: minor | Milestone: Component: GetComponents | Version: Keywords: | ---------------------------+------------------------------------------------ In a CRL file, it should be possible to have only an AUTH_URL field and omit the URL field, since it might be that there is no unauthenticated way to access the repository (e.g. for private repositories). At the moment, when I omit the URL field for a Git repository, the error message is: {{{ Use of uninitialized value $git_repo in substitution (s///) at ./GetComponents line 589. Use of uninitialized value $git_repo in substitution (s///) at ./GetComponents line 590. Use of uninitialized value $rec{"GIT_REPO"} in substitution (s///) at ./GetComponents line 593. Use of uninitialized value $rec{"GIT_REPO"} in substitution (s///) at ./GetComponents line 594. ... }}} -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1045> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #1772: Simfactory: potentially serious problem with CACHE directory in the simulations directory
by Einstein Toolkit 31 Aug '15

31 Aug '15

#1772: Simfactory: potentially serious problem with CACHE directory in the simulations directory ------------------------+--------------------------------------------------- Reporter: bmundim | Owner: Type: defect | Status: new Priority: critical | Milestone: ET_2015_05 Component: SimFactory | Version: development version Keywords: CACHE | ------------------------+--------------------------------------------------- Is the directory CACHE in the simulation directory really necessary? We are talking about executables with at most 400MB of size, which is nothing compared to current HPC storage systems. I think I might have found a design flaw on simfactory use of CACHE directory which can go unnoticed until it is too late with potential loss of thousands of SUs. Suppose we have the following situation: 1) We build a configuration A and send a simulation A1 with with parameter file 1. So simfactory copies the executable from configuration A to simulation A1 simfactory directory and creates a symlink from /scratch/simulations/CACHE/exe/cactus_A to /scratch/simulations/A1/SIMFACTORY/exe/cactus_A. 2) We then create a new simulation A2 with a different parameter file 2. This time simfactory symlink the simulation executable /scratch/simulations/A2/SIMFACTORY/exe/cactus_A to the cached one /scratch/simulations/CACHE/exe/cactus_A. 3) After a few days (or restarts) of simulations A1 and A2, you come up with a better idea/fix/new parameter which requires to recompile your configuration A. Note that we don't want to build a new configuration from scratch since cactus configurations consume both a lot of time and space to build. So you rebuild your configuration A and its executable cactus_A is updated. 4) Let's say now we submit the updated configuration with the same parameter file 2 in order to test your new idea/fix/parameter and compare it with the simulation A2, which is still running and have a few extra restarts to completion. Call this simulation A2_updated. Simfactory then copy the new updated executable cactus_A from the Cactus/exe/cactus_A to the simulation directory /scratch/simulations/A2_updated/SIMFACTORY/exe/cactus_A *and* update the CACHE symlink to that new simulation directory, ie: $ cd /scratch/simulations/CACHE/exe $ ls -l cactus_A cactus_A -> ../../../../scratch/simulations/A2_updated/SIMFACTORY/exe/cactus_A 5) The problem: now my simulation A2 restarts are compromised with a new executable. Remember that that simulation executable is actually a symlink to the one in the CACHE directory, which has just been updated. I think this whole cache directory intermediate step introduces unnecessary complexity for the user to track; it is really unnecessary and in my opinion not a good design choice. I would vote to eliminate it from simfactory completely as soon as possible, ideally even for this release. Just use one copy of the executable from cactus/exe to simulation/SIMFACTORY/exe and that's it. This is all we need to have that simulation and future ones running consistently with the same executable. Thanks! PS: I have actually noticed this issue on Hershel release (there is no option pointing to Hershel release on trac). I am working on tests for development version to confirm this issue, but give simfactory commits I believe it is still there. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1772> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 25

← Newer
1
...
6
7
8
9
10
11
12
Older →

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Trac August 2015