[Users] Extracting the last iteration from 3D CarpetIOHDF5 output files

[Lorenzo Ennoggi]

Hi Roland,
thank you very much for your answer, I made it using hdf5_slicer. I'm
puzzled though, because in my Python script I copied each dataset into a
separate file, so I was expecting nioprocs = 160 to be correct (there are
160 files). Anyway, hdf5_slicer worked perfectly.

Thank you again,
Lorenzo

Il giorno lun 27 giu 2022 alle ore 15:20 Roland Haas <rhaas at illinois.edu>
ha scritto:

> Hello Lorenzo,
>
> There's a metadata nioprocs or so that is rhe number of files. If you
> copied all datasets into a single file you must set it to 1. You can use a
> tiny Python script to do so, or (I think) hdf5_slicer --out3d_box, or (I
> think) hdf5_merge with its --ioprocs option.
>
> In fact you can use hdf5_slicer to do the extraction for you. There's some
> details about this in the response to an earlier question by Maria about
> merging files I think.
>
> Yours,
> Roland
>
> ----- Original Message -----
> From: Lorenzo Ennoggi <lorenzo.ennoggi at gmail.com>
> Sent: 2022-06-27 - 07:01
> To: Einstein Toolkit Users <users at einsteintoolkit.org>
> Subject: [Users] Extracting the last iteration from 3D CarpetIOHDF5 output
> files
>
> > Hello,
> > I have a set of 3D output files from CarpetIOHDF5 (rho.xyz_file<n>.h5),
> > each containing many iterations, and I want to create a set of new
> > CarpetIOHDF5 files containing the data related to the last iteration. I
> > have tried something, but I couldn't get anything to work, so I am asking
> > for help here.
> >
> > For each file rho.xyz_file<n>.h5, I used the hdf5_extract utility to
> create
> > a new file containing the dataset related to the last iteration
> > ("HYDROBASE::rho
> > it=1067600 tl=0 rl=0 c=<n>") and the group "Parameters and Global
> > Attributes". However, when I try to open the set of new files with VisIt
> I
> > get the following error:
> >
> > VisIt could not read from the file
> >>
> "/home/lorenzo/CBD_prod_WZ9_400_140_280_output-0014/Output/rho.xyz_it1067600_tl0_rl0_c*
> >> database".
> >
> >
> > The generated error message was:
> >
> >
> > There was an error opening
> >>
> /home/lorenzo/CBD_prod_WZ9_400_140_280_output-0014/Output/rho.xyz_it1067600_tl0_rl0_c*
> >> database. It may be an invalid file.  VisIt tried using the following
> file
> >> format readers to open the file: CarpetHDF5
> >
> >
> > The following error(s) may be helpful in identifying the problem:
> >> *Tried to access an invalid index 1 (Maximum = 0).*
> >
> >
> > I thought that maybe some metadata in the new files are telling VisIt
> that
> > there are multiple iterations available, which is not true; I'm not
> > completely sure this is really the issue, though.
> >
> > In the new files, the attributes "GH$iteration" and "carpet_global_time"
> > from the group "Parameters and Global Attributes" are still respectively
> > set to the first iteration (992800) and the first time (148920) available
> > in the original files, while I am extracting the last iteration (1067600,
> > time 160140). Also, in the new files, the dataset "Grid Structure v5" (a
> > string) still contains
> > "grid_times:[[[148920,148919.85000000001,148919.69999999998]]]", which
> also
> > looks wrong. Therefore, I used h5py to generate new files in which I
> fixed
> > "GH$iteration", "carpet_global_time" and "Grid Structure v5", but I still
> > get the same error from VisIt.
> >
> > I have also thought about re-running the simulation that produced the
> > original files from the last checkpoint and just dump the last iteration,
> > but that simulation was run some time ago and the code I am using has
> > evolved quite a bit in the meantime, to the point that some parameters
> are
> > not even defined anymore.
> >
> > Is there any other attribute/dataset I should edit in order to fix the
> new
> > files? Are there smarter ways to extract an iteration from a set of 3D
> > CarpetHDF5 output files? Please let me know if you need additional
> > information from my end.
> >
> > Thank you very much,
> > Lorenzo Ennoggi
> > _______________________________________________
> > Users mailing list
> > Users at einsteintoolkit.org
> >
> https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!8ek7C4tvxIysQIy8c3UhecmG8i6HaqZW_HTD-WMedhlEHkX7_FBP7ORbQoGjMqNr2SXd2ekz8pJr7kHksP2S8ShBJg$
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20220627/e07e30ad/attachment-0001.html