Trac September 2015

trac@lists.einsteintoolkit.org

1 participants
94 discussions

[Einstein Toolkit] #1033: support a combined "map" file in CarpetIOHDF5
by Einstein Toolkit 07 Oct '15

07 Oct '15

#1033: support a combined "map" file in CarpetIOHDF5 --------------------------+------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: enhancement | Status: new Priority: minor | Milestone: Component: Carpet | Version: Keywords: CarpetIOHDF5 | --------------------------+------------------------------------------------- recovering on different number of processes than was used to write a checkpoint is painfully slow. Part of the reason seems to be that each process essentially has to read all files to find out where each piece of data it requires is located. The attached patch (not to be included in the main code due to bad file formats and coding) enables CarpetIOHDF5 to read all the information stored in the union of index files to from a single file. This means (together with the other patches proposed today) that CarpetIOHDF5 only ever opens those HDF5 files that are required to restore the simulation on a given process. It significantly (factor > 4 where I don't quite know how fast since the unpatched version ran out of walltime) speeds up recovery with many more processors than wrote the files. It also adds an optimization for CCTK_VarIndex calls inside CarpetIOHDF5 (which happens for every dataset in the file). This is intended only as a proof of what might speed up recovery. A proper implementation would need a more sensible file format. Two option seem possible: 1) extend the index file format by a "filename" or "filenum" attribute to each dataset and use a concatenation of all index files as the map file 2) define a custom hdf5 data type corresponding to the information in a single patch_t, which would have mostly integer field plus two variable length / enumerated ASCII fields (for the patch name, variable name) -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1033> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #1737: upate CarpetHDF5 reader in VisIt
by Einstein Toolkit 15 Sep '15

15 Sep '15

#1737: upate CarpetHDF5 reader in VisIt ------------------------------+--------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: Other | Version: development version Keywords: VisIt CarpetHDF5 | ------------------------------+--------------------------------------------- Since the CarpetHDF5 reader (https://svn.cactuscode.org/VizTools/CarpetHDF5) was included in VisIt's main source code repo, we have not attempted to update that copy to the most current version. The coding style in that branch matches the official code but the patches do not apply on top of the official code due to ordering issues as well as some minor new changes to code formatting in the current (2.8.0) VisIt codebase. Since then, I have collected a number of bugfixes/improvements that are collected in various branches at https://bitbucket.org/rhaas80/carpethdf5 . It would be good to eventually try and get the for_VisIt branch (https://bitbucket.org/rhaas80/carpethdf5/branch/for_VisIt) included in the main source code repo again. It contains some bug-fixes wrt how file metadata is cached, as well as number of fixes to reduce memory footprint, number of open files (required to work on large datasets on machines that limit the total number of open files), some improvements to error reporting and robustness when dealing with partially corrupted filesets as well as changes that allow a user to combine multiple HDF5 files into a single "virtual" VisIt database using .visit files. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1737> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #1813: Change mechanism for comparing data in test suites
by Einstein Toolkit 14 Sep '15

14 Sep '15

#1813: Change mechanism for comparing data in test suites -------------------------+-------------------------------------------------- Reporter: sbrandt | Owner: Type: enhancement | Status: new Priority: optional | Milestone: Component: Cactus | Version: development version Keywords: | -------------------------+-------------------------------------------------- This test changes the way test data is compared. Instead of comparing line by line, it forms a key of the (t,x,y,z) coordinates and compares the values at those coordinates. Consequently, I also disable the check that the required number of procs are used. This allows us to have one set of test files, generated on a single proc, and run the test suite with multiple procs. This should make the disk space occupied by tests smaller, and make the existing tests more flexible. Pull request: https://bitbucket.org/cactuscode/cactus/pull-requests/18 /modify-the-test-utilities-so-that-a-test/diff -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1813> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #1045: URL field should be optional
by Einstein Toolkit 01 Sep '15

01 Sep '15

#1045: URL field should be optional ---------------------------+------------------------------------------------ Reporter: hinder | Owner: eric9 Type: defect | Status: new Priority: minor | Milestone: Component: GetComponents | Version: Keywords: | ---------------------------+------------------------------------------------ In a CRL file, it should be possible to have only an AUTH_URL field and omit the URL field, since it might be that there is no unauthenticated way to access the repository (e.g. for private repositories). At the moment, when I omit the URL field for a Git repository, the error message is: {{{ Use of uninitialized value $git_repo in substitution (s///) at ./GetComponents line 589. Use of uninitialized value $git_repo in substitution (s///) at ./GetComponents line 590. Use of uninitialized value $rec{"GIT_REPO"} in substitution (s///) at ./GetComponents line 593. Use of uninitialized value $rec{"GIT_REPO"} in substitution (s///) at ./GetComponents line 594. ... }}} -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1045> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Trac September 2015