[Users] VisIt HDF5 directory-dependent read-in failures?

[Bernard Kelly]

Hi Roland.

Thanks! I rejected option 1, as I still want to combine these with
other files later, so having a "database" of chunks is too awkward.

Nevertheless, option 2 --- hdf5_slicer with the syntax you suggested
--- seems to have fixed my problem

Bernard

On 26 July 2016 at 11:53, Roland Haas <rhaas at aei.mpg.de> wrote:
> Hello Bernard,
>
> hdf5_merge will not work with Carpet's chunked output. It does
> everything correctly with the exception of not updating the nioprocs
> attribute of "Parameters and Global Attributes"
>
> Bascially what I think happened was that you opened the "merged" file
> but VisIt kept looking for the .file_1.h5 etc files which it found as
> long as the merged file was in the same directory as the original files.
>
> You now have three options:
>
> 1. just open XX.file_0.h5 and VisIt will automatically open file_1.h5
> etc.
> 2. use hdf5_slicer --out3d-cube --match '.*' XX.file_*.h5 XX.h5 to
> merge the files
> 3. fix the merged file using the attached python script.
>
> 4. use hdf5_merge from
> https://github.com/rhaas80/fileutils/tree/master/hdf5tools using its -u (unchunk) option.
>
> Yours,
> Roland
>
>> Hi all.
>>
>> I have some Carpet-generated 3D HDF5 data I want to look at in VisIt,
>> but have encountered a weird issue trying to read it in, and wondered
>> whether anyone else has encountered it.
>>
>> The data output is from a single time step, but was chunked, so there
>> are 32 separate files that have to be recombined to reconstruct the
>> whole data snapshot. I'm using the hdf5_merge utility for this
>> purpose. Here's the issue:
>>
>> * if I start up VisIt and read the merged data file from the directory
>> it was created in, it works fine.
>>
>> * if I move/copy the merged data file elsewhere (like down a level in
>> the directory tree), and try to get VisIt to read from the new
>> location, it fails. Sample error (I tried gluing together just the
>> first two chunks for this example):
>>
>> ------------------------------------------------------------------------------------------------
>> VisIt could not read from the file "<full directory path>/By.xyz.file_0-1.h5".
>>
>> The generated error message was:
>>
>> There was an error opening <full directory path>/By.xyz.file_0-1.h5.
>> It may be an invalid file.  VisIt tried using the following file
>> format readers to open the file: CarpetHDF5
>>
>> The following error(s) may be helpful in identifying the problem:
>> Tried to access an invalid index 1 (Maximum = 0).
>> ------------------------------------------------------------------------------------------------
>>
>> I have no problem with opening any of the original chunked files,
>> regardless of location.
>>
>> Has anyone else seen anything like this?
>>
>> Thanks,
>>
>> Bernard
>>
>> P.S. After starting to compose this, I found that *some* of the
>> maerged files started working everywhere, while others didn't. It's
>> quite confusing.
>>
>> P.P.S. This is the latest version of VisIt, installed on OS X 10.11.6
>> (El Capitan). I don't know which version of ET produced the original
>> data, but my hdf5_merge came from ET two generations back (2015_05).
>> _______________________________________________
>> Users mailing list
>> Users at einsteintoolkit.org
>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
>
>
> --
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://keys.gnupg.net.