Trac April 2011

trac@lists.einsteintoolkit.org

1 participants
139 discussions

[Einstein Toolkit] #180: Reduce disk space used by checkpoints
by Einstein Toolkit 11 Jul '11

11 Jul '11

#180: Reduce disk space used by checkpoints -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: enhancement | Status: new Priority: major | Milestone: Component: SimFactory | Version: Keywords: | -------------------------+-------------------------------------------------- Currently simfactory stores the checkpoints for each restart in their own directory. This means that the Cactus mechanism for deleting all but the most recent N checkpoints does not see the previous checkpoints. This means that you can easily run out of quota space when doing very long simulations. One solution to this would be for SimFactory to store the checkpoints in a directory above the output-NNNN directories and make the current checkpoints directories under output-NNNN symbolic links to the common directory. This way, all restarts would see the same directory for checkpoint files, and Cactus could clean up the old checkpoints. The current hardlinking mechanism would not be required any more. This solution might be undesirable because it means that each restart is no longer independent. Another solution would be to give simfactory an option to delete old checkpoint files from previous restarts when a job starts. This would duplicate the functionality already available in Cactus. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/180> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 8

[Einstein Toolkit] #285: python simfactory submit does not remove active link after job execution has finished
by Einstein Toolkit 11 Jul '11

11 Jul '11

#285: python simfactory submit does not remove active link after job execution has finished ----------------------------------------------+----------------------------- Reporter: alexander.beck-ratzka@… | Type: defect Status: new | Priority: major Milestone: | Component: Other Version: | Keywords: ----------------------------------------------+----------------------------- The python version of simfactory does not remove the active link output-xxxx-active in the simulation directory, after a job has finished. As a consequence the next submit fails with the error message, that more then one active link has been found. Removing the link manually enables a poper submit of the next job. The problem occurs on a lustre file system. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/285> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 7

[Einstein Toolkit] #419: CarpetRegrid2 sets up grids which are not consistent
by Einstein Toolkit 06 Jul '11

06 Jul '11

#419: CarpetRegrid2 sets up grids which are not consistent --------------------+------------------------------------------------------- Reporter: knarf | Owner: eschnett Type: defect | Status: new Priority: minor | Milestone: Component: Carpet | Version: Keywords: | --------------------+------------------------------------------------------- The attached parfile creates a grid using CarpetRegrid2 (hg version), which CarpetReduce later warns about: INFO (CarpetReduce): Simulation domain volume: 4096 INFO (CarpetReduce): Reduction weight sum: 4468.125 WARNING[L1,P0] (CarpetReduce): Simulation domain volume and reduction weight sum differ The grid info is: INFO (Carpet): Grid structure (superregions, grid points): [0][0][0] exterior: [0,0,0] : [21,21,21] ([22,22,22] + PADDING) 10648 [1][0][0] exterior: [3,3,3] : [40,40,40] ([38,38,38] + PADDING) 54872 [2][0][0] exterior: [9,9,9] : [42,42,42] ([34,34,34] + PADDING) 39304 [3][0][0] exterior: [21,21,21] : [52,52,52] ([32,32,32] + PADDING) 32768 [4][0][0] exterior: [45,45,45] : [72,72,72] ([28,28,28] + PADDING) 21952 INFO (Carpet): Grid structure (superregions, coordinates): [0][0][0] exterior: [-48,-48,-48] : [288,288,288] : [16,16,16] [1][0][0] exterior: [-24,-24,-24] : [272,272,272] : [8,8,8] [2][0][0] exterior: [-12,-12,-12] : [120,120,120] : [4,4,4] [3][0][0] exterior: [-6,-6,-6] : [56,56,56] : [2,2,2] [4][0][0] exterior: [-3,-3,-3] : [24,24,24] : [1,1,1] -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/419> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 14

[Einstein Toolkit] #381: Cannot log in to Kraken using default simfactory configuration
by Einstein Toolkit 06 Jul '11

06 Jul '11

#381: Cannot log in to Kraken using default simfactory configuration ------------------------+--------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: new Priority: major | Milestone: ET_2011_05 Component: SimFactory | Version: Keywords: | ------------------------+--------------------------------------------------- SimFactory uses the gsissh method to connect to Kraken. This requires a "proxy" to be created locally via the myproxy-logon command. There is currently logic in Perl simfactory's mdb.pm file to automatically run myproxy-logon and prompt for the passphrase. This does not work for me. I get the error Failed to receive credentials. ERROR from myproxy-server (myproxy.teragrid.org): PAM authentication failed: Permission denied (see below for the full output). If I remove the localsshsetup key, I can run the myproxy-logon command manually and then connect using simfactory. Looking at the full output below, it seems that it is trying to run myproxy-logon with my local username (ian) instead of the remote one (hinder). If I replace the @USER@ in the myproxy-logon command with my Kraken username, everything works as expected. MacBook:etrelease ian$ sim login kraken Simulation Factory: Executing: { mkdir -p /Users/ian/Cactus/etrelease/.globus && : >> /Users/ian/Cactus/etrelease/.globus/proxy-teragrid && chmod go-rwx /Users/ian/Cactus/etrelease/.globus/proxy-teragrid && export X509_USER_PROXY=/Users/ian/Cactus/etrelease/.globus/proxy-teragrid && : mkdir -p /Users/ian/Cactus/etrelease/.globus/certificates-teragrid && : export X509_CERT_DIR=/Users/ian/Cactus/etrelease/.globus/certificates- teragrid && { { grid-proxy-info -issuer -file /Users/ian/Cactus/etrelease/.globus /proxy-teragrid 2>/dev/null | grep "^/C=US/O=National Center for Supercomputing Applications/" >/dev/null 2>/dev/null && test $(grid-proxy-info -timeleft -file /Users/ian/Cactus/etrelease/.globus/proxy-teragrid 2>/dev/null) -gt 0 2>/dev/null } || { { grid-proxy-destroy 2>/dev/null || true; } && myproxy-logon -p 7514 -s myproxy.teragrid.org -T -l ian -o /Users/ian/Cactus/etrelease/.globus/proxy-teragrid } } && { globus-update-certificate-dir > /dev/null 2>&1 || true; }; } && gsissh -t hinder(a)kraken-gsi2.nics.teragrid.org '{ source /etc/profile; } && cd /nics/b/home/hinder/Cactus/etrelease && $SHELL -l' Enter MyProxy pass phrase: Failed to receive credentials. ERROR from myproxy-server (myproxy.teragrid.org): PAM authentication failed: Permission denied -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/381> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #113: Simfactory should be able to run the Cactus testsuites
by Einstein Toolkit 05 Jul '11

05 Jul '11

#113: Simfactory should be able to run the Cactus testsuites -------------------------+-------------------------------------------------- Reporter: knarf | Owner: mthomas Type: enhancement | Status: new Priority: critical | Milestone: ET_2011_06 Component: SimFactory | Version: Keywords: | -------------------------+-------------------------------------------------- ... in the python version, before the next ET release. The scripts at http://einsteintoolkit.org/release-info/ might help with implementing that. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/113> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 7

[Einstein Toolkit] #372: Accept comments in multi-line strings in parameter files
by Einstein Toolkit 02 Jul '11

02 Jul '11

#372: Accept comments in multi-line strings in parameter files -------------------------+-------------------------------------------------- Reporter: eschnett | Owner: Type: enhancement | Status: new Priority: major | Milestone: Component: Cactus | Version: Keywords: | -------------------------+-------------------------------------------------- The attached patch modifies the parameter file parser to accept comments in multi-line strings. These comments extend from a "#" character to the end of the line. This makes it much easier to comment out variables in output strings. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/372> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #349: pyc files when syncing
by Einstein Toolkit 23 Jun '11

23 Jun '11

#349: pyc files when syncing ---------------------------+------------------------------------------------ Reporter: barry.wardell | Owner: mthomas Type: defect | Status: new Priority: major | Milestone: Component: SimFactory | Version: Keywords: | ---------------------------+------------------------------------------------ When I use 'sim sync' to copy files to a machine, it often rsyncs several .pyc files in the SimFactory directory. This happens every time I sync after running a simfactory command on the remote machine. I think these files should probably be excluded from the sync process. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/349> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 16

[Einstein Toolkit] #316: Checkpoint recovery nonfunctional
by Einstein Toolkit 23 May '11

23 May '11

#316: Checkpoint recovery nonfunctional ------------------------+--------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: new Priority: blocker | Milestone: Component: SimFactory | Version: Keywords: regression | ------------------------+--------------------------------------------------- Checkpoint recovery is nonfunctional in SimFactory 2 (it has broken since it was last fixed in ticket #60). Using the attached parameter file, I submit a simulation on Datura: simfactory2/bin/sim --machine datura --config sim2_datura create-submit parfiles/cptest.par 12 1:00:00 This parameter file terminates the Cactus run after 1 minute and dumps a checkpoint file. I then manually remove the output-0000-active symlink, as the automatic cleanup in the main() function is cleaning up restarts that are attempting to run, so I have disabled it, and manual cleanup doesn't work (see ticket #315). I then resubmit the simulation simfactory2/bin/sim --machine datura submit parfiles/cptest.par and observe that the checkpoint files from the first restart are never hardlinked into the output directory. The job does not recover, and instead starts from initial data. Log file is attached. Looking at the code, it appears that the checkpoint linking is conditional on the from-restart-id parameter being passed to simfactory, which I think is something to do with job-chaining. I can't see anywhere in the code which sets this option, so this is probably why the linking is not happening. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/316> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 13

[Einstein Toolkit] #344: Checkpointing + new cleanup procedure
by Einstein Toolkit 23 May '11

23 May '11

#344: Checkpointing + new cleanup procedure -------------------------+-------------------------------------------------- Reporter: mthomas | Owner: mthomas Type: enhancement | Status: new Priority: blocker | Milestone: Component: SimFactory | Version: Keywords: | -------------------------+-------------------------------------------------- Attached to this ticket is the patch to fix checkpointing (which Ian has already reviewed) plus code implementing the new cleanup procedure. The new cleanup procedure is thus: 1. Automatic cleanup of every simulation is now gone. 2. Any command that creates a new restart (submit, user initiated run) calls restartlib.CleanupSimulation on that specific simulation, and CleanupSimulation will only attempt to do cleanup if it finds an active restart for that given simulation 3. sim cleanup without any arguments will cleanup all simulations. If you specify a specific simulation, it will only clean up that one. Still needing to be implemented are the times when cleanup of all simulations should happen, or the cleanup when a simulation finishes. These need to be done via cronjobs or some other method that hasn't quite been figured out yet. Please svn up to get the latest revision (I committed a bunch of very trivial code changes) then apply this patch and add comments to this ticket related to this patch. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/344> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 8

[Einstein Toolkit] #354: Patch: Do not clean up all simulations all the time
by Einstein Toolkit 23 May '11

23 May '11

#354: Patch: Do not clean up all simulations all the time -------------------------+-------------------------------------------------- Reporter: eschnett | Owner: mthomas Type: enhancement | Status: new Priority: major | Milestone: Component: SimFactory | Version: Keywords: | -------------------------+-------------------------------------------------- The enclosed patch disables the global cleanup calll, and instead cleans up individual simulations as they are submitted. It also removes the "finished" attribute of restarts, which is not really necessary since one can instead look for the "*-active" symlink. It re-organises the code internally somewhat, as the current code assumes that restarts in the "U" (finished) state do not need to be cleaned up any more. In addition, it adds a set of TODO comments where the current code could be improved. This patch probably conflicts severely with Michael Thomas's patch addressing the same issue; I propose to combine both patches in some way before committing them. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/354> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 1

← Newer
1
...
6
7
8
9
10
11
12
13
14
Older →

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Trac April 2011