#1878: hdf5 deflate fragments memory --------------------------+------------------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: Other | Version: development version Resolution: | Keywords: HDF5 --------------------------+-------------------------------------------------
Comment (by hinder):
I have used tcmalloc at the same time as the HDF5 decompression, and this was not sufficient to alleviate the problem. It might be that the HDF5 library was not using the tcmalloc malloc (I think I asked you about this in the past Erik; I will have to check the email conversation), or the fragmentation was just too much, so that even tcmalloc was not able to cope. I think that the problem originates in the HDF5 decompression routine, and could be made a lot better by a few small changes there. We should find out whether the size of the original data field can be used or not.