[ET Trac] [Einstein Toolkit] #860: Don't rmtree(scratchdir)

Einstein Toolkit trac-noreply at einsteintoolkit.org
Thu May 3 12:40:15 CDT 2012


#860: Don't rmtree(scratchdir)
----------------------------+-----------------------------------------------
  Reporter:  barry.wardell  |       Owner:  eschnett
      Type:  defect         |      Status:  new     
  Priority:  blocker        |   Milestone:          
 Component:  SimFactory     |     Version:          
Resolution:                 |    Keywords:          
----------------------------+-----------------------------------------------

Comment (by eschnett):

 The scratch directory is a directory that "belongs" to the simulation. It
 is made available at the beginning and is removed at the end, so that
 unused files don't accumulate. Saving disk space is important in many
 cases. This also allows using facilities where e.g. the queuing system
 deletes scratch directories, or where the scratch directories live on the
 nodes are become unavailable after the run finishes.

 I am sorry if there is a misunderstanding about the meaning of this MDB
 entry. Yes, setting the scratch directory to "/scratch" makes only sense
 if these are node-local directories.

 In most cases, the scratch directory contains e.g. the job id, as in
 {{{
 /scratch/scratchdirs/@USER@/scratch/@JOB_ID@
 }}}
 to guarantee that it is unique.

 One solution could be to check whether the scratch directory is empty
 before a job starts, or to even ensure that it does not exist. If it
 exists this would be a fatal error, pointing to a significant
 inconsistency in the MDB or the system setup.

 Another option would be to deprecate scratchdir (since at least one person
 misunderstood its meaning), and introduce scratchbasedir instead. The
 scratchbasedir is supposed to exist, and the scratchdir is created as a
 subdirectory (that must not yet exist). This subdirectory will then be
 deleted again when the restart finishes.

 If scratchbasedir is not set, a subdirectory of the simulation directory
 would be used. Setting scratchbasedir will allow using a faster file
 system if one is available.

 In the mean time: Please tell me which machine this is, so that we can
 correct its MDB.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/860#comment:1>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit


More information about the Trac mailing list