[ET Trac] [Einstein Toolkit] #860: Don't rmtree(scratchdir)
Einstein Toolkit
trac-noreply at einsteintoolkit.org
Thu May 3 12:40:15 CDT 2012
#860: Don't rmtree(scratchdir)
----------------------------+-----------------------------------------------
Reporter: barry.wardell | Owner: eschnett
Type: defect | Status: new
Priority: blocker | Milestone:
Component: SimFactory | Version:
Resolution: | Keywords:
----------------------------+-----------------------------------------------
Comment (by eschnett):
The scratch directory is a directory that "belongs" to the simulation. It
is made available at the beginning and is removed at the end, so that
unused files don't accumulate. Saving disk space is important in many
cases. This also allows using facilities where e.g. the queuing system
deletes scratch directories, or where the scratch directories live on the
nodes are become unavailable after the run finishes.
I am sorry if there is a misunderstanding about the meaning of this MDB
entry. Yes, setting the scratch directory to "/scratch" makes only sense
if these are node-local directories.
In most cases, the scratch directory contains e.g. the job id, as in
{{{
/scratch/scratchdirs/@USER@/scratch/@JOB_ID@
}}}
to guarantee that it is unique.
One solution could be to check whether the scratch directory is empty
before a job starts, or to even ensure that it does not exist. If it
exists this would be a fatal error, pointing to a significant
inconsistency in the MDB or the system setup.
Another option would be to deprecate scratchdir (since at least one person
misunderstood its meaning), and introduce scratchbasedir instead. The
scratchbasedir is supposed to exist, and the scratchdir is created as a
subdirectory (that must not yet exist). This subdirectory will then be
deleted again when the restart finishes.
If scratchbasedir is not set, a subdirectory of the simulation directory
would be used. Setting scratchbasedir will allow using a faster file
system if one is available.
In the mean time: Please tell me which machine this is, so that we can
correct its MDB.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/860#comment:1>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
More information about the Trac
mailing list