[ET Trac] [Einstein Toolkit] #1878: hdf5 deflate fragments memory

[Einstein Toolkit]

#1878: hdf5 deflate fragments memory
--------------------------+-------------------------------------------------
  Reporter:  knarf        |       Owner:                     
      Type:  enhancement  |      Status:  new                
  Priority:  minor        |   Milestone:                     
 Component:  Other        |     Version:  development version
Resolution:               |    Keywords:  HDF5               
--------------------------+-------------------------------------------------

Comment (by eschnett):

 I wonder whether memory fragmentation is not a problem caused by the
 malloc implementation. To test this, you can test with different (newer)
 versions of libc, or with a malloc replacement. I recommend in particular
 jemalloc <http://www.canonware.com/jemalloc/> -- you build the code as
 usual, and add {{{-ljemalloc}}} when linking. This replaces malloc, free,
 etc., and the equivalent C++ mechanism. jemalloc claims to be better than
 GNU malloc is many ways.

 Another way to avoid fragmentation (that we use in CarpetIOASCII) is to
 allocate memory not directly via malloc, but rather from a pre-allocated
 pool. This pool is so large that fragmentation is not an issue on 64-bit
 systems, since it can be returned to the OS as a whole. Implementing a
 pool is not too difficult; see Carpet's mempool class (CarpetLib, mem.hh
 and mem.cc).

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1878#comment:1>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit