[Users] New parallel HDF5 API proposed

[Erik Schnetter]

We're now building HDF5 ourselves with Cactus, since this ensures that
we have up-to-date versions (with e.g. all Lustre performance
improvements) that work with the particular compilers and MPI versions
we use.

Unfortunately, we just decided to support the C++ interface for HDF5,
and this is incompatible with the parallel HDF5 interface. I wonder
why -- maybe that'll change in HDF5 1.10.

-erik

On Wed, Sep 29, 2010 at 10:11 AM, Werner Benger <werner at cct.lsu.edu> wrote:
> I have not seen a concrete release date, I doubt it would already
> be part of 1.10. As of now they're looking for feedback on the envisioned
> API before starting to implement it, so very early state yet.
>
> However, as far as I understand it, it will be based on parallel HDF5.
> Cactus doesn't use parallel HDF5, but serial HDF5. It might have been
> a decade ago since Thomas Radke tried parallel HDF5 with Cactus, so that
> would want to be re-animated, requiring MPI-IO to be installed on the
> respective machines as prerequisite. That apparently was an issue years
> ago, maybe it's no longer one.
>
>        Werner
>
>
> On Wed, 29 Sep 2010 09:19:46 -0500, Erik Schnetter <schnetter at cct.lsu.edu>
> wrote:
>
>> I think this is only a proposal at the moment. I think HDF 1.10 is
>> currently under development -- maybe it'll make it in there, maybe
>> it'll be 1.12. I'm not following HDF5 development, but Werner Benger
>> does, maybe he knows.
>>
>> -erik
>>
>> On Wed, Sep 29, 2010 at 2:28 AM, Michael Koppitz <koppitz at aei.mpg.de>
>> wrote:
>>>
>>> This is intriguing indeed!
>>> When will this new API be publicly available?
>>>
>>>
>>> Michael
>>>
>>>
>>> On Sep 29, 2010, at 3:24 AM, Erik Schnetter wrote:
>>>
>>>> The HDF5 groups is apparently contemplating adding a new high-level
>>>> API for "HPC HDF5", based on parallel HDF5. I find the idea
>>>> intriguing, as it would move much code that we have currently written
>>>> in Carpet (and that we will have to write to address the "slow
>>>> recovery" issue) into HDF5. In particular, recovering would consist
>>>> essentially of a single call to H5HPCdataset_read_mult_all(), which
>>>> would read multiple datasets from multiple processes.
>>>>
>>>> -erik
>>>>
>>>> --
>>>> Erik Schnetter <schnetter at cct.lsu.edu>
>>>> http://www.cct.lsu.edu/~eschnett/
>>>>
>>>> <H5HPC_RFC-2010-09-28.pdf>_______________________________________________
>>>> Users mailing list
>>>> Users at einsteintoolkit.org
>>>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at einsteintoolkit.org
>>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>>>
>>
>>
>>
>
>
> --
> ___________________________________________________________________________
> Dr. Werner Benger                Visualization Research
> Laboratory for Creative Arts and Technology (LCAT)
> Center for Computation & Technology at Louisiana State University (CCT/LSU)
> 211 Johnston Hall, Baton Rouge, Louisiana 70803
> Tel.: +1 225 578 4809                        Fax.: +1 225 578-5362
>



-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/