<div dir="ltr">Hi Roland,<div>indeed changing the names of the files I'm creating with my Python script to foo.file_<n>.h5 solved the issue and I can correctly visualize the data with Visit.</div><div><br></div><div>Thanks a lot again,</div><div>Lorenzo</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il giorno lun 27 giu 2022 alle ore 17:15 Roland Haas <<a href="mailto:rhaas@illinois.edu">rhaas@illinois.edu</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello Lorenzo,<br>
<br>
Python can be tricky. niprocs must be a 32bit integer and Python's int<br>
type by default is 64bit and it tends to re-create the attribute rather<br>
than change it. So depending on how you copied the attributes group<br>
this may affect you.<br>
<br>
My own nioprocs.py script thus looks like this (note the int32 cast):<br>
<br>
--8<--<br>
#!/usr/bin/python<br>
<br>
import h5py<br>
import sys<br>
import numpy<br>
<br>
cfh = h5py.File(sys.argv[1], "r+")<br>
cfh.require_group("Parameters and Global Attributes")<br>
cfh["Parameters and Global Attributes"].attrs["nioprocs"] = numpy.int32(1)<br>
--8<--<br>
<br>
Or it could be somthing with the file names not quite matching the form<br>
the VisIt reader expects (must be either foo.file_X.h5 or foo.h5 if<br>
there is a single ioproc).<br>
<br>
Anyway, glad that this worked for you.<br>
<br>
Yours,<br>
Roland<br>
<br>
> Hi Roland,<br>
> thank you very much for your answer, I made it using hdf5_slicer. I'm<br>
> puzzled though, because in my Python script I copied each dataset into a<br>
> separate file, so I was expecting nioprocs = 160 to be correct (there are<br>
> 160 files). Anyway, hdf5_slicer worked perfectly.<br>
> <br>
> Thank you again,<br>
> Lorenzo<br>
> <br>
> Il giorno lun 27 giu 2022 alle ore 15:20 Roland Haas <<a href="mailto:rhaas@illinois.edu" target="_blank">rhaas@illinois.edu</a>><br>
> ha scritto:<br>
> <br>
> > Hello Lorenzo,<br>
> ><br>
> > There's a metadata nioprocs or so that is rhe number of files. If you<br>
> > copied all datasets into a single file you must set it to 1. You can use a<br>
> > tiny Python script to do so, or (I think) hdf5_slicer --out3d_box, or (I<br>
> > think) hdf5_merge with its --ioprocs option.<br>
> ><br>
> > In fact you can use hdf5_slicer to do the extraction for you. There's some<br>
> > details about this in the response to an earlier question by Maria about<br>
> > merging files I think.<br>
> ><br>
> > Yours,<br>
> > Roland<br>
> ><br>
> > ----- Original Message -----<br>
> > From: Lorenzo Ennoggi <<a href="mailto:lorenzo.ennoggi@gmail.com" target="_blank">lorenzo.ennoggi@gmail.com</a>><br>
> > Sent: 2022-06-27 - 07:01<br>
> > To: Einstein Toolkit Users <<a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a>><br>
> > Subject: [Users] Extracting the last iteration from 3D CarpetIOHDF5 output<br>
> > files<br>
> > <br>
> > > Hello,<br>
> > > I have a set of 3D output files from CarpetIOHDF5 (rho.xyz_file<n>.h5),<br>
> > > each containing many iterations, and I want to create a set of new<br>
> > > CarpetIOHDF5 files containing the data related to the last iteration. I<br>
> > > have tried something, but I couldn't get anything to work, so I am asking<br>
> > > for help here.<br>
> > ><br>
> > > For each file rho.xyz_file<n>.h5, I used the hdf5_extract utility to <br>
> > create <br>
> > > a new file containing the dataset related to the last iteration<br>
> > > ("HYDROBASE::rho<br>
> > > it=1067600 tl=0 rl=0 c=<n>") and the group "Parameters and Global<br>
> > > Attributes". However, when I try to open the set of new files with VisIt <br>
> > I <br>
> > > get the following error:<br>
> > ><br>
> > > VisIt could not read from the file <br>
> > >> <br>
> > "/home/lorenzo/CBD_prod_WZ9_400_140_280_output-0014/Output/rho.xyz_it1067600_tl0_rl0_c* <br>
> > >> database". <br>
> > ><br>
> > ><br>
> > > The generated error message was:<br>
> > ><br>
> > ><br>
> > > There was an error opening <br>
> > >> <br>
> > /home/lorenzo/CBD_prod_WZ9_400_140_280_output-0014/Output/rho.xyz_it1067600_tl0_rl0_c* <br>
> > >> database. It may be an invalid file. VisIt tried using the following <br>
> > file <br>
> > >> format readers to open the file: CarpetHDF5 <br>
> > ><br>
> > ><br>
> > > The following error(s) may be helpful in identifying the problem: <br>
> > >> *Tried to access an invalid index 1 (Maximum = 0).* <br>
> > ><br>
> > ><br>
> > > I thought that maybe some metadata in the new files are telling VisIt <br>
> > that <br>
> > > there are multiple iterations available, which is not true; I'm not<br>
> > > completely sure this is really the issue, though.<br>
> > ><br>
> > > In the new files, the attributes "GH$iteration" and "carpet_global_time"<br>
> > > from the group "Parameters and Global Attributes" are still respectively<br>
> > > set to the first iteration (992800) and the first time (148920) available<br>
> > > in the original files, while I am extracting the last iteration (1067600,<br>
> > > time 160140). Also, in the new files, the dataset "Grid Structure v5" (a<br>
> > > string) still contains<br>
> > > "grid_times:[[[148920,148919.85000000001,148919.69999999998]]]", which <br>
> > also <br>
> > > looks wrong. Therefore, I used h5py to generate new files in which I <br>
> > fixed <br>
> > > "GH$iteration", "carpet_global_time" and "Grid Structure v5", but I still<br>
> > > get the same error from VisIt.<br>
> > ><br>
> > > I have also thought about re-running the simulation that produced the<br>
> > > original files from the last checkpoint and just dump the last iteration,<br>
> > > but that simulation was run some time ago and the code I am using has<br>
> > > evolved quite a bit in the meantime, to the point that some parameters <br>
> > are <br>
> > > not even defined anymore.<br>
> > ><br>
> > > Is there any other attribute/dataset I should edit in order to fix the <br>
> > new <br>
> > > files? Are there smarter ways to extract an iteration from a set of 3D<br>
> > > CarpetHDF5 output files? Please let me know if you need additional<br>
> > > information from my end.<br>
> > ><br>
> > > Thank you very much,<br>
> > > Lorenzo Ennoggi<br>
> > > _______________________________________________<br>
> > > Users mailing list<br>
> > > <a href="mailto:Users@einsteintoolkit.org" target="_blank">Users@einsteintoolkit.org</a><br>
> > > <br>
> > <a href="https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!8ek7C4tvxIysQIy8c3UhecmG8i6HaqZW_HTD-WMedhlEHkX7_FBP7ORbQoGjMqNr2SXd2ekz8pJr7kHksP2S8ShBJg$" rel="noreferrer" target="_blank">https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!8ek7C4tvxIysQIy8c3UhecmG8i6HaqZW_HTD-WMedhlEHkX7_FBP7ORbQoGjMqNr2SXd2ekz8pJr7kHksP2S8ShBJg$</a><br>
> ><br>
> ><br>
> > <br>
<br>
-- <br>
My email is as private as my paper mail. I therefore support encrypting<br>
and signing email messages. Get my PGP key from <a href="http://pgp.mit.edu" rel="noreferrer" target="_blank">http://pgp.mit.edu</a> .<br>
</blockquote></div>