Trac February 2015

trac@lists.einsteintoolkit.org

1 participants
109 discussions

[Einstein Toolkit] #1033: support a combined "map" file in CarpetIOHDF5
by Einstein Toolkit 07 Oct '15

07 Oct '15

#1033: support a combined "map" file in CarpetIOHDF5 --------------------------+------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: enhancement | Status: new Priority: minor | Milestone: Component: Carpet | Version: Keywords: CarpetIOHDF5 | --------------------------+------------------------------------------------- recovering on different number of processes than was used to write a checkpoint is painfully slow. Part of the reason seems to be that each process essentially has to read all files to find out where each piece of data it requires is located. The attached patch (not to be included in the main code due to bad file formats and coding) enables CarpetIOHDF5 to read all the information stored in the union of index files to from a single file. This means (together with the other patches proposed today) that CarpetIOHDF5 only ever opens those HDF5 files that are required to restore the simulation on a given process. It significantly (factor > 4 where I don't quite know how fast since the unpatched version ran out of walltime) speeds up recovery with many more processors than wrote the files. It also adds an optimization for CCTK_VarIndex calls inside CarpetIOHDF5 (which happens for every dataset in the file). This is intended only as a proof of what might speed up recovery. A proper implementation would need a more sensible file format. Two option seem possible: 1) extend the index file format by a "filename" or "filenum" attribute to each dataset and use a concatenation of all index files as the map file 2) define a custom hdf5 data type corresponding to the information in a single patch_t, which would have mostly integer field plus two variable length / enumerated ASCII fields (for the patch name, variable name) -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1033> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #1737: upate CarpetHDF5 reader in VisIt
by Einstein Toolkit 15 Sep '15

15 Sep '15

#1737: upate CarpetHDF5 reader in VisIt ------------------------------+--------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: Other | Version: development version Keywords: VisIt CarpetHDF5 | ------------------------------+--------------------------------------------- Since the CarpetHDF5 reader (https://svn.cactuscode.org/VizTools/CarpetHDF5) was included in VisIt's main source code repo, we have not attempted to update that copy to the most current version. The coding style in that branch matches the official code but the patches do not apply on top of the official code due to ordering issues as well as some minor new changes to code formatting in the current (2.8.0) VisIt codebase. Since then, I have collected a number of bugfixes/improvements that are collected in various branches at https://bitbucket.org/rhaas80/carpethdf5 . It would be good to eventually try and get the for_VisIt branch (https://bitbucket.org/rhaas80/carpethdf5/branch/for_VisIt) included in the main source code repo again. It contains some bug-fixes wrt how file metadata is cached, as well as number of fixes to reduce memory footprint, number of open files (required to work on large datasets on machines that limit the total number of open files), some improvements to error reporting and robustness when dealing with partially corrupted filesets as well as changes that allow a user to combine multiple HDF5 files into a single "virtual" VisIt database using .visit files. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1737> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #1045: URL field should be optional
by Einstein Toolkit 01 Sep '15

01 Sep '15

#1045: URL field should be optional ---------------------------+------------------------------------------------ Reporter: hinder | Owner: eric9 Type: defect | Status: new Priority: minor | Milestone: Component: GetComponents | Version: Keywords: | ---------------------------+------------------------------------------------ In a CRL file, it should be possible to have only an AUTH_URL field and omit the URL field, since it might be that there is no unauthenticated way to access the repository (e.g. for private repositories). At the moment, when I omit the URL field for a Git repository, the error message is: {{{ Use of uninitialized value $git_repo in substitution (s///) at ./GetComponents line 589. Use of uninitialized value $git_repo in substitution (s///) at ./GetComponents line 590. Use of uninitialized value $rec{"GIT_REPO"} in substitution (s///) at ./GetComponents line 593. Use of uninitialized value $rec{"GIT_REPO"} in substitution (s///) at ./GetComponents line 594. ... }}} -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1045> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #1749: avoid creating temporary links to files in Formaline
by Einstein Toolkit 18 Aug '15

18 Aug '15

#1749: avoid creating temporary links to files in Formaline -----------------------------------+---------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Keywords: Formaline | -----------------------------------+---------------------------------------- the patch in https://bitbucket.org/cactuscode/cactusutils/branch/rhaas %2Fupdate-index changes the way Formaline populates the configjar git repository with changed files. Instead of making a hard linked copy of every file in the thorn, it used plumbing commands git-update-index to add them to the index. This deals gracefully with files in symbolically linked directories (but will dereference a symbolic link if the file *itself* is a symgolic link) and also works when CACTUS_CONFIGS_DIR is not on the same file system as the source tree (eg Cactus is in $HOME which has a small quota but is backed up and $CACTUS_CONFIGS_DIR is in $SCRATCH which is not backed up). Not being able to have $CACTUS_CONFIGS_DIR on a different file system than Cactus is a minor bug, in particular since this method is described in the UserGuide. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1749> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 8

[Einstein Toolkit] #334: unsuccessful qsub not recognized / submit succeeds for finished simulation
by Einstein Toolkit 18 Aug '15

18 Aug '15

#334: unsuccessful qsub not recognized / submit succeeds for finished simulation ------------------------+--------------------------------------------------- Reporter: knarf | Owner: mthomas Type: defect | Status: new Priority: minor | Milestone: Component: SimFactory | Version: Keywords: | ------------------------+--------------------------------------------------- python version: I submitted a simulation 'sim submit' but the corresponding qsub failed due to wrong numbers of procs/node (philip cluster). I changed the number given on the command line and did a 'sim submit' again, this time successful. Several things happend which I think could be done better: - the unsuccessful qsub was not detected during the new submit - it attempted a restart and didn't simply clean the unsuccessful submit - when trying the restart, it went ahead and queued the job, but this later failed when run with "cannot rerun a restart that has been finished". This could have been caught earlier - without the wait time in the queue. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/334> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 6

[Einstein Toolkit] #1627: Merge rewrite branch of McLachlan
by Einstein Toolkit 14 Aug '15

14 Aug '15

#1627: Merge rewrite branch of McLachlan -----------------------------------+---------------------------------------- Reporter: eschnett | Owner: Type: enhancement | Status: new Priority: major | Milestone: Component: EinsteinToolkit thorn | Version: development version Keywords: | -----------------------------------+---------------------------------------- We should merge the "rewrite" branch of McLachlan. To do before this merge: - Remove helper thorn - Add backward compatibility layer for parameters -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1627> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 5

[Einstein Toolkit] #649: simfactory should create the "simulations" directory
by Einstein Toolkit 14 Aug '15

14 Aug '15

#649: simfactory should create the "simulations" directory ------------------------+--------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: defect | Status: new Priority: minor | Milestone: Component: SimFactory | Version: Keywords: | ------------------------+--------------------------------------------------- when following the new users tutorial one gets: {{{ [rhaas@qb1 Cactus]$ ./simfactory/bin/sim submit static_tov --parfile=par/static_tov.par --procs=32 --walltime=8:0:0 Parameter file: /home/rhaas/Cactus/par/static_tov.par Error: could not access simulation base directory /scratch/rhaas/simulations for reading and writing Aborting Simfactory. [137450 refs] }}} This happens each time I set up a fresh simfactory on a machine. It might be useful if simfactory would create the simulation base directory if it does not exist. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/649> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 2

[Einstein Toolkit] #1645: simfactory should create exe directory
by Einstein Toolkit 14 Aug '15

14 Aug '15

#1645: simfactory should create exe directory ------------------------+--------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: defect | Status: new Priority: minor | Milestone: Component: SimFactory | Version: development version Keywords: | ------------------------+--------------------------------------------------- Using virtual (--virtual) configuration one can wrap a Cactus config (or at least the parts that simfactory cares about) around an existing executable. Simfactory then copies the executable to exe/cactus_sim (or whatever), however it assumes that exe already exists. Since exe is not part of the Cactus checkout, simfactory needs to itself ensure that it exists. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1645> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 4

[Einstein Toolkit] #1615: TOVSolver shares parameters from GRHydro
by Einstein Toolkit 13 Aug '15

13 Aug '15

#1615: TOVSolver shares parameters from GRHydro -----------------------------------+---------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Keywords: TOVSolver | -----------------------------------+---------------------------------------- Zach Etienne was looking at TOVSolver to use it for a test case in IllinoisGRMHD (and to compare to GRHydro). He found that it Turns out all but one of the parameters that are shared with GRHydro are unused in TOVSolver: These need to be removed: USES real rho_abs_min USES real rho_rel_min USES REAL initial_rho_abs_min USES REAL initial_rho_rel_min USES REAL initial_atmosphere_factor and that the one shared parameter that is actually used is GRHydro_rho_central. My suggestion would to remove these dependencies so that the initial data thorn can be used with other evolution thorns. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1615> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #1059: simfactory silent error
by Einstein Toolkit 13 Aug '15

13 Aug '15

#1059: simfactory silent error --------------------------------------+------------------------------------- Reporter: anonymous | Owner: eschnett Type: enhancement | Status: new Priority: optional | Milestone: Component: SimFactory | Version: Keywords: simfactory error message | --------------------------------------+------------------------------------- When sim submit -remote ... fails to queue a job because the allocation has been overdrawn, simfactory does not print any error message to screen (it is only hidden in the log file). Could it be made to print an error message to screen? -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1059> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

← Newer
1
...
5
6
7
8
9
10
11
Older →

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Trac February 2015