[Users] New parallel HDF5 API proposed

[Werner Benger]

In my experience, the HDF5 C++ API is not very convincing. It was  
implemented
based on HDF5 1.6.1 and not much progress had been made to keep track with
more recent feature. For instance, it implements HDF5 C++ objects via  
reference
counting, which is already done by HDF5 itself, but was not exposed in the  
API
until 1.6.2. So HDF5 C++ does double work here. Also, it makes extensive  
usage
of C++ exceptions, which C++ users complained about that exceptions would  
not
be designed for such usage. In the end, most C++ users seem to use the C  
API
and have their own C++ wrappers around HDF5. I haven't observed any  
significant
work on HDF5 C++ in the last years, so don't have the impression there will
happen much for 1.10 .

	Werner


On Wed, 29 Sep 2010 12:42:51 -0500, Erik Schnetter <schnetter at cct.lsu.edu>  
wrote:

> We're now building HDF5 ourselves with Cactus, since this ensures that
> we have up-to-date versions (with e.g. all Lustre performance
> improvements) that work with the particular compilers and MPI versions
> we use.
>
> Unfortunately, we just decided to support the C++ interface for HDF5,
> and this is incompatible with the parallel HDF5 interface. I wonder
> why -- maybe that'll change in HDF5 1.10.
>
> -erik
>
> On Wed, Sep 29, 2010 at 10:11 AM, Werner Benger <werner at cct.lsu.edu>  
> wrote:
>> I have not seen a concrete release date, I doubt it would already
>> be part of 1.10. As of now they're looking for feedback on the  
>> envisioned
>> API before starting to implement it, so very early state yet.
>>
>> However, as far as I understand it, it will be based on parallel HDF5.
>> Cactus doesn't use parallel HDF5, but serial HDF5. It might have been
>> a decade ago since Thomas Radke tried parallel HDF5 with Cactus, so that
>> would want to be re-animated, requiring MPI-IO to be installed on the
>> respective machines as prerequisite. That apparently was an issue years
>> ago, maybe it's no longer one.
>>
>>        Werner
>>
>>
>> On Wed, 29 Sep 2010 09:19:46 -0500, Erik Schnetter  
>> <schnetter at cct.lsu.edu>
>> wrote:
>>
>>> I think this is only a proposal at the moment. I think HDF 1.10 is
>>> currently under development -- maybe it'll make it in there, maybe
>>> it'll be 1.12. I'm not following HDF5 development, but Werner Benger
>>> does, maybe he knows.
>>>
>>> -erik
>>>
>>> On Wed, Sep 29, 2010 at 2:28 AM, Michael Koppitz <koppitz at aei.mpg.de>
>>> wrote:
>>>>
>>>> This is intriguing indeed!
>>>> When will this new API be publicly available?
>>>>
>>>>
>>>> Michael
>>>>
>>>>
>>>> On Sep 29, 2010, at 3:24 AM, Erik Schnetter wrote:
>>>>
>>>>> The HDF5 groups is apparently contemplating adding a new high-level
>>>>> API for "HPC HDF5", based on parallel HDF5. I find the idea
>>>>> intriguing, as it would move much code that we have currently written
>>>>> in Carpet (and that we will have to write to address the "slow
>>>>> recovery" issue) into HDF5. In particular, recovering would consist
>>>>> essentially of a single call to H5HPCdataset_read_mult_all(), which
>>>>> would read multiple datasets from multiple processes.
>>>>>
>>>>> -erik
>>>>>
>>>>> --
>>>>> Erik Schnetter <schnetter at cct.lsu.edu>
>>>>> http://www.cct.lsu.edu/~eschnett/
>>>>>
>>>>> <H5HPC_RFC-2010-09-28.pdf>_______________________________________________
>>>>> Users mailing list
>>>>> Users at einsteintoolkit.org
>>>>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at einsteintoolkit.org
>>>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>>>>
>>>
>>>
>>>
>>
>>
>> --
>> ___________________________________________________________________________
>> Dr. Werner Benger                Visualization Research
>> Laboratory for Creative Arts and Technology (LCAT)
>> Center for Computation & Technology at Louisiana State University  
>> (CCT/LSU)
>> 211 Johnston Hall, Baton Rouge, Louisiana 70803
>> Tel.: +1 225 578 4809                        Fax.: +1 225 578-5362
>>
>
>
>


-- 
___________________________________________________________________________
Dr. Werner Benger                Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809                        Fax.: +1 225 578-5362