[ET Trac] [Einstein Toolkit] #1743: Reduce number of output files per directory

Einstein Toolkit trac-noreply at einsteintoolkit.org
Mon Feb 9 16:42:37 CST 2015

#1743: Reduce number of output files per directory
  Reporter:  eschnett  |       Owner:                     
      Type:  defect    |      Status:  new                
  Priority:  unset     |   Milestone:                     
 Component:  Other     |     Version:  development version
Resolution:            |    Keywords:                     

Comment (by eschnett):

 My idea was to group things my process id, since this reduces I/O traffic:
 each directory would only be accessed from a subset of all processes.
 Whether a process writes one or multiple files into a directory is likely
 a secondary effect due to caching.

 Having one directory per process may not be enough, since running on 10k
 processes then still creates 10k sub-directories in one directory. I was
 thinking of a hierarchical approach:

 where proc00nnnn is used by the first 10k processes, proc0000nn is used by
 the first 100 processes, etc.

Ticket URL: <https://trac.einsteintoolkit.org/ticket/1743#comment:2>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit

More information about the Trac mailing list