[Users] Merging carpet hdf5 data without creating additional files

Tue May 17 14:08:18 CDT 2016

It is surprising to see how many such frameworks are available. I'd suggest
that we create a wiki page that briefly describes them, and points to
instructions for downloading and examples.

Incidentally, we are also developing such a framework here at Perimeter.
The basic idea is to have a generic way of describing the data in a
simulation, introducing concepts such as discretizations of a manifold,
bases for the tangent space, tensor types, fields, etc. <
https://github.com/eschnett/SimulationIO>. We (i.e. Jonah Miller) are now
testing and benchmarking it against real-world data, developing a yt <
http://yt-project.org> reader in the process.

The SimulationIO library is supposed to be used directly for writing and
reading data from Cactus. Currently, that hasn't been implemented yet, and
so there is a converter from Carpet output that either collects all the
metadata (with external links to the original datasets) or creates a single
output file. Obviously the file format is still based on HDF5 -- the main
advantage is that the metadata are much easier to access and interpret than
having to loop over the components and parsing their names.

I assume Jonah will describe this at the Einstein Toolkit.

-erik

On Tue, May 17, 2016 at 2:17 PM, Federico Guercilena <
guercilena at th.physik.uni-frankfurt.de> wrote:

> Hello,
>
> I'd like to point out also my python library/script called "rugutils".
> This one started off inspired by David's scidata (and pygraph), but has
> developed into a set of tools to look in a quick and hassle-free way at
> Carpet data. It's written in python 3.5
>
> https://bitbucket.org/fguercilena/rugutils
> https://docs.einsteintoolkit.org/et-docs/Analysis_and_post-processing
>
> Federico
>
> 2016-05-17 19:44 GMT+02:00 Roland Haas <rhaas at aei.mpg.de>:
>
>> Hello Frank, Michael,
>>
>> @Michael: both of what Frank or David describe should work. If you know
>> how to compile the VisIt reader yourselv, you can also try the copy of
>> CarpetHDF5 at https://bitbucket.org/rhaas80/carpethdf5 either the
>> "multifile" branch (which is the default one that has all features) or
>> the "for_VisIt" branch which has fewer features and is what I wanted at
>> one point propose for inclusion in VisIt. Both will let you
>> create .visit files listing your hdf5 files and VisIt will treat them a
>> a single database. You will have to create .visit files that look like
>> this:
>>
>> !NBLOCKS 3
>> output-0000/rho.file_0.h5
>> output-0000/rho.file_1.h5
>> output-0000/rho.file_2.h5
>> output-0001/rho.file_0.h5
>> output-0001/rho.file_1.h5
>> output-0001/rho.file_2.h5
>>
>> for a 3 process run and similar for runs with more processes. You can
>> probably (I have not tested this) also list both rho and eps:
>>
>> !NBLOCKS 3
>> output-0000/rho.file_0.h5
>> output-0000/rho.file_1.h5
>> output-0000/rho.file_2.h5
>> output-0000/eps.file_0.h5
>> output-0000/eps.file_1.h5
>> output-0000/eps.file_2.h5
>> output-0001/rho.file_0.h5
>> output-0001/rho.file_1.h5
>> output-0001/rho.file_2.h5
>> output-0001/eps.file_0.h5
>> output-0001/eps.file_1.h5
>> output-0001/eps.file_2.h5
>>
>> possibly you have to set NBLOCKS to 6 (2*3) for this, though I have not
>> tested
>> this either.
>>
>> @Frank:
>> > Also, in case you have an old version of VisIt, you might need to enable
>> > the use of soft links in the VisIt Carpet reader. I am currently not
>> > sure if that change made it back upstream yet. The corresponding patch
>> > is:
>> >
>> > Index: avtCarpetHDF5FileFormat.C
>> > ===================================================================
>> > --- avtCarpetHDF5FileFormat.C   (revision 57)
>> > +++ avtCarpetHDF5FileFormat.C   (working copy)
>> > @@ -1044,7 +1044,7 @@
>> >     sprintf(fullname, "%s%s%s", rootname,
>> rootname[strlen(rootname)-1]=='/' ? "" : "/",member_name);
>> >
>> >     // we are interested in datasets only - skip anything else
>> > -   H5Gget_objinfo (group_id, member_name, 0, &object_info);
>> > +   H5Gget_objinfo (group_id, member_name, 1, &object_info);
>> >     if (object_info.type != H5G_DATASET)
>> >     {
>> >        if (object_info.type == H5G_GROUP)
>> As far as I can tell this code change has not even made it into the ET
>> maintained copy of the reader:
>> https://svn.cactuscode.org/VizTools/CarpetHDF5/
>>
>> where if this does not cause any other problems I think it would be
>> welcome as well.
>>
>> Yours,
>> Roland
>>
>>
>>
>> --
>> My email is as private as my paper mail. I therefore support encrypting
>> and signing email messages. Get my PGP key from http://keys.gnupg.net.
>>
>> _______________________________________________
>> Users mailing list
>> Users at einsteintoolkit.org
>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>>
>>
>
>
> --
> Federico Guercilena
> Institut für Theoretische Physik
> Johann Wolfgang Goethe-Universität
> Max-von-Laue-Str. 1
> 60438 Frankfurt am Main, Germany
> Telephone: +49 69 798 47887
> Email: guercilena[at]th.physik.uni-frankfurt.de
>
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
>

-- 
Erik Schnetter <schnetter at cct.lsu.edu>
http://www.perimeterinstitute.ca/personal/eschnetter/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20160517/9ff34ea4/attachment.html 