Trac March 2011

trac@lists.einsteintoolkit.org

1 participants
136 discussions

[Einstein Toolkit] #316: Checkpoint recovery nonfunctional
by Einstein Toolkit 23 May '11

23 May '11

#316: Checkpoint recovery nonfunctional ------------------------+--------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: new Priority: blocker | Milestone: Component: SimFactory | Version: Keywords: regression | ------------------------+--------------------------------------------------- Checkpoint recovery is nonfunctional in SimFactory 2 (it has broken since it was last fixed in ticket #60). Using the attached parameter file, I submit a simulation on Datura: simfactory2/bin/sim --machine datura --config sim2_datura create-submit parfiles/cptest.par 12 1:00:00 This parameter file terminates the Cactus run after 1 minute and dumps a checkpoint file. I then manually remove the output-0000-active symlink, as the automatic cleanup in the main() function is cleaning up restarts that are attempting to run, so I have disabled it, and manual cleanup doesn't work (see ticket #315). I then resubmit the simulation simfactory2/bin/sim --machine datura submit parfiles/cptest.par and observe that the checkpoint files from the first restart are never hardlinked into the output directory. The job does not recover, and instead starts from initial data. Log file is attached. Looking at the code, it appears that the checkpoint linking is conditional on the from-restart-id parameter being passed to simfactory, which I think is something to do with job-chaining. I can't see anywhere in the code which sets this option, so this is probably why the linking is not happening. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/316> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 13

[Einstein Toolkit] #344: Checkpointing + new cleanup procedure
by Einstein Toolkit 23 May '11

23 May '11

#344: Checkpointing + new cleanup procedure -------------------------+-------------------------------------------------- Reporter: mthomas | Owner: mthomas Type: enhancement | Status: new Priority: blocker | Milestone: Component: SimFactory | Version: Keywords: | -------------------------+-------------------------------------------------- Attached to this ticket is the patch to fix checkpointing (which Ian has already reviewed) plus code implementing the new cleanup procedure. The new cleanup procedure is thus: 1. Automatic cleanup of every simulation is now gone. 2. Any command that creates a new restart (submit, user initiated run) calls restartlib.CleanupSimulation on that specific simulation, and CleanupSimulation will only attempt to do cleanup if it finds an active restart for that given simulation 3. sim cleanup without any arguments will cleanup all simulations. If you specify a specific simulation, it will only clean up that one. Still needing to be implemented are the times when cleanup of all simulations should happen, or the cleanup when a simulation finishes. These need to be done via cronjobs or some other method that hasn't quite been figured out yet. Please svn up to get the latest revision (I committed a bunch of very trivial code changes) then apply this patch and add comments to this ticket related to this patch. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/344> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 8

[Einstein Toolkit] #354: Patch: Do not clean up all simulations all the time
by Einstein Toolkit 23 May '11

23 May '11

#354: Patch: Do not clean up all simulations all the time -------------------------+-------------------------------------------------- Reporter: eschnett | Owner: mthomas Type: enhancement | Status: new Priority: major | Milestone: Component: SimFactory | Version: Keywords: | -------------------------+-------------------------------------------------- The enclosed patch disables the global cleanup calll, and instead cleans up individual simulations as they are submitted. It also removes the "finished" attribute of restarts, which is not really necessary since one can instead look for the "*-active" symlink. It re-organises the code internally somewhat, as the current code assumes that restarts in the "U" (finished) state do not need to be cleaned up any more. In addition, it adds a set of TODO comments where the current code could be improved. This patch probably conflicts severely with Michael Thomas's patch addressing the same issue; I propose to combine both patches in some way before committing them. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/354> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #304: SimFactory should not clean-up so aggressively
by Einstein Toolkit 23 May '11

23 May '11

#304: SimFactory should not clean-up so aggressively ------------------------+--------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: new Priority: major | Milestone: Component: SimFactory | Version: Keywords: | ------------------------+--------------------------------------------------- As far as I can tell, SimFactory 2 currently cleans up all simulations every time it is run. This is not scalable - if you are running with a slow production filesystem, just statting all the simulations could take a very long time. Similarly, if you do a sim sync, it currently cleans up all the simulations on your local machine, as well as the remote machine. Running sim --help even cleans up all your simulations! This behaviour is very counter-intuitive (certainly not what a user would expect) and not suitable for production systems. I propose that simfactory should only clean up the simulation which is being referred to in the current command. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/304> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #328: TRAC frequently logs users out
by Einstein Toolkit 19 May '11

19 May '11

#328: TRAC frequently logs users out ----------------------------------+----------------------------------------- Reporter: hinder | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit trac | Version: Keywords: | ----------------------------------+----------------------------------------- There seems to be some mechanism which is automatically logging users out of TRAC. This means that they end up posting as "anonymous" because they don't realise they are logged out. Probably there is a session cookie which expires or something. To avoid this, we could either try to change the logout (I would be happy if the automatic logout was just disabled completely), or make it much more obvious when you are not logged in. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/328> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 7

[Einstein Toolkit] #348: https://docs.einsteintoolkit.org/ does not redirect to the docs page
by Einstein Toolkit 18 May '11

18 May '11

#348: https://docs.einsteintoolkit.org/ does not redirect to the docs page -------------------------------------+-------------------------------------- Reporter: hinder | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit website | Version: Keywords: docs | -------------------------------------+-------------------------------------- When I go to https://docs.einsteintoolkit.org/ it takes me to the list of CCT wikis. Based on the hostname, it should redirect to the correct page automatically. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/348> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

[Einstein Toolkit] #242: build fails after running out of disk space
by Einstein Toolkit 02 May '11

02 May '11

#242: build fails after running out of disk space ------------------------+--------------------------------------------------- Reporter: eschnett | Owner: mthomas Type: defect | Status: new Priority: major | Milestone: Component: SimFactory | Version: Keywords: | ------------------------+--------------------------------------------------- After running out of disk space (and then removing 16 GByte), building fails on numrel09: $ ./bin/sim build DEBUG: Simfactory command: ./bin/../simfactory/lib/sim.py "build" DEBUG: Version exported The Simulation Factory: Manage Cactus simulations Info: defs: /home/eschnett/EinsteinToolkit-hg- vanilla/simfactory/etc/defs.ini Info: defs.local: /home/eschnett/EinsteinToolkit-hg- vanilla/simfactory/etc/defs.local.ini Info: Executing command: build Using configuration: sim Info: Cactus Directory: /home/eschnett/EinsteinToolkit-hg-vanilla Traceback (most recent call last): File "./bin/../simfactory/lib/sim.py", line 142, in <module> main() File "./bin/../simfactory/lib/sim.py", line 139, in main CommandDispatch() File "./bin/../simfactory/lib/sim.py", line 106, in CommandDispatch module.main() File "/home/eschnett/EinsteinToolkit-hg-vanilla/simfactory/lib/sim- build.py", line 420, in main CommandDispatch() File "/home/eschnett/EinsteinToolkit-hg-vanilla/simfactory/lib/sim- build.py", line 347, in CommandDispatch exec("command_%s()" % command) File "<string>", line 1, in <module> File "/home/eschnett/EinsteinToolkit-hg-vanilla/simfactory/lib/sim- build.py", line 391, in command_build PrepConfiguration(config) File "/home/eschnett/EinsteinToolkit-hg-vanilla/simfactory/lib/sim- build.py", line 58, in PrepConfiguration prop.init(propertiesFile) File "/home/eschnett/EinsteinToolkit-hg- vanilla/simfactory/lib/restartlib.py", line 101, in init self.ImportProperties() File "/home/eschnett/EinsteinToolkit-hg- vanilla/simfactory/lib/restartlib.py", line 107, in ImportProperties for key in section.keys(): AttributeError: 'NoneType' object has no attribute 'keys' This looks as if some internal information about the build has become corrupt, maybe because of a partially written file. This may be a rare case not worth handling elegantly. However -- what should I do now? -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/242> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 7

[Einstein Toolkit] #363: Make TimerReport output average/min/max timings to stdout, instead of timings from the root process
by Einstein Toolkit 30 Apr '11

30 Apr '11

#363: Make TimerReport output average/min/max timings to stdout, instead of timings from the root process ----------------------+----------------------------------------------------- Reporter: eschnett | Owner: Type: defect | Status: new Priority: major | Milestone: Component: Cactus | Version: Keywords: | ----------------------+----------------------------------------------------- The n_top_timers feature of TimerReport outputs timings from the root process only to stdout, greatly confusing people. It should instead output average (or min/max) timings over all processes. One common problem is a routine that contains communication, and which thus has to wait until all processes arrive there. If previous routines show a load imbalance, then outputting timings only from the root process make this routine look very slow, although this routine really only has to wait for other processes finish their previous calculations. Since this is a common case, it makes the current timing output useless for all routines involving communication. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/363> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 5

[Einstein Toolkit] #313: Reorder CUDA specific makefile rules
by Einstein Toolkit 22 Apr '11

22 Apr '11

#313: Reorder CUDA specific makefile rules -------------------------+-------------------------------------------------- Reporter: eschnett | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: Cactus | Version: Keywords: | -------------------------+-------------------------------------------------- The rules to build CUDA source files are very similar to those for C and C++, and different from those for the Fortrans. I suggest to reorder these makefile rules to make the code easier to understand. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/313> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 2

[Einstein Toolkit] #268: The CUDA flags need to be documented
by Einstein Toolkit 19 Apr '11

19 Apr '11

#268: The CUDA flags need to be documented -------------------------+-------------------------------------------------- Reporter: knarf | Owner: jtao Type: enhancement | Status: new Priority: minor | Milestone: Component: Cactus | Version: Keywords: | -------------------------+-------------------------------------------------- The new CUDA flags in the flesh (r4684) need to be documented. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/268> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 4

← Newer
1
...
5
6
7
8
9
10
11
...
14
Older →

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Trac March 2011