[Users] HDF5 Error, Hilbert

[Ian Hinder]

On 5 Oct 2015, at 12:43, Geraint Pratten <g.pratten at sussex.ac.uk> wrote:

> Hi,
> 
> I am trying to run a simulation with the same output directory - though I've been removing the simulations after each run while trying to fix the above issue. I should have write access to the file and there should be sufficient storage. 

Can you check the standard output of the run and see if the number of processes and threads are what you expect?  You should see something like "Carpet is running on N processes with M threads".  Maybe there is a problem in MPI initialisation, and multiple processes are trying to write to the same HDF5 file at the same time.


> 
> Geraint
> 
> On 5 October 2015 at 11:34, Ian Hinder <ian.hinder at aei.mpg.de> wrote:
> 
> On 1 Oct 2015, at 15:49, Erik Schnetter <schnetter at cct.lsu.edu> wrote:
> 
>> Geraint
>> 
>> Some very fundamental HDF5 operations fail. This could be due to a file system problem; maybe you don't have write access to the file, or your disk is full. It could also be that there exists a corrupted HDF5 file, and you are trying to write to it (extend it).
>> 
>> HDF5 files can get corrupted if you have opened it for writing, and then the application is interrupted or crashes, or you run out of disk space. Even if you make free space later, a corrupted file remains corrupted.
> 
> Hi Geraint,
> 
> Are you trying to run a simulation with output in the same directory as one that was run before?  
> 
>> 
>> -erik
>> 
>> 
>> On Thu, Oct 1, 2015 at 9:30 AM, Geraint Pratten <g.pratten at sussex.ac.uk> wrote:
>> Hi all,
>> 
>> I recently built the latest release (Hilbert) on the local cluster and I've been getting a HDF5 error. I've tried building Cactus with both the local HDF5 package on the cluster as well as letting Cactus build HDF5 from scratch. Does anyone have any insight into the error I've attached below (extract of a long chain of errors of the same form)?
>> 
>> I can dump the output to file or give configuration files if needed.
>> 
>> Thanks in advance!
>> Geraint
>> 
>> ----------
>> 
>> HDF5-DIAG: Error detected in HDF5 (1.8.14) thread 0:
>>   #000: H5Ddeprec.c line 193 in H5Dcreate1(): unable to create dataset
>>     major: Dataset
>>     minor: Unable to initialize object
>>   #001: H5Dint.c line 453 in H5D__create_named(): unable to create and link to dataset
>>     major: Dataset
>>     minor: Unable to initialize object
>>   #002: H5L.c line 1638 in H5L_link_object(): unable to create new link to object
>>     major: Links
>>     minor: Unable to initialize object
>>   #003: H5L.c line 1882 in H5L_create_real(): can't insert link
>>     major: Symbol table
>>     minor: Unable to insert object
>>   #004: H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
>>     major: Symbol table
>>     minor: Object not found
>>   #005: H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
>>     major: Symbol table
>>     minor: Callback failed
>>   #006: H5L.c line 1674 in H5L_link_cb(): name already exists
>>     major: Symbol table
>>     minor: Object already exists
>> WARNING level 1 from host node207.cm.cluster process 0
>>   while executing schedule bin (none), routine (no thorn)::(no routine)
>>   in thorn CarpetIOHDF5, file /mnt/pact/gp234/NR/Cactus/arrangements/Carpet/CarpetIOHDF5/src/Output.cc:677:
>>   -> HDF5 call 'dataset = H5Dcreate (outfile, datasetname.str().c_str(), filedatatype, dataspace, plist)' returned error code -1
>> WARNING level 1 from host node207.cm.cluster process 0
>>   while executing schedule bin (none), routine (no thorn)::(no routine)
>>   in thorn CarpetIOHDF5, file /mnt/pact/gp234/NR/Cactus/arrangements/Carpet/CarpetIOHDF5/src/Output.cc:689:
>>   -> HDF5 call 'H5Dwrite (dataset, memdatatype, H5S_ALL, H5S_ALL, H5P_DEFAULT, data)' returned error code -1
>> WARNING level 1 from host node207.cm.cluster process 0
>>   while executing schedule bin (none), routine (no thorn)::(no routine)
>>   in thorn CarpetIOHDF5, file /mnt/pact/gp234/NR/Cactus/arrangements/Carpet/CarpetIOHDF5/src/Output.cc:725:
>>   -> HDF5 call 'attr = H5Acreate (dataset, "level", H5T_NATIVE_INT, dataspace, H5P_DEFAULT)' returned error code -1
>> WARNING level 1 from host node207.cm.cluster process 0
>>   while executing schedule bin (none), routine (no thorn)::(no routine)
>>   in thorn CarpetIOHDF5, file /mnt/pact/gp234/NR/Cactus/arrangements/Carpet/CarpetIOHDF5/src/Output.cc:726:
>>   -> HDF5 call 'H5Awrite (attr, H5T_NATIVE_INT, &refinementlevel)' returned error code -1
>> WARNING level 1 from host node207.cm.cluster process 0
>> 
>> -- 
>> Geraint Pratten
>> Postdoctoral Research Associate
>> 
>> Mobile: +44(0) 7581709282
>> E-mail: G.Pratten at sussex.ac.uk
>> Skype: geraint.pratten
>> 
>> School of Mathematical and Physical Sciences
>> Pevensey 3 Building
>> University of Sussex
>> Falmer Campus
>> Brighton
>> BN1 9QH
>> United Kingdom
>> 
>> _______________________________________________
>> Users mailing list
>> Users at einsteintoolkit.org
>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>> 
>> 
>> 
>> 
>> -- 
>> Erik Schnetter <schnetter at cct.lsu.edu>
>> http://www.perimeterinstitute.ca/personal/eschnetter/
>> _______________________________________________
>> Users mailing list
>> Users at einsteintoolkit.org
>> http://lists.einsteintoolkit.org/mailman/listinfo/users
> 
> -- 
> Ian Hinder
> http://members.aei.mpg.de/ianhin
> 
> 
> 
> 
> -- 
> Geraint Pratten
> Postdoctoral Research Associate
> 
> Mobile: +44(0) 7581709282
> E-mail: G.Pratten at sussex.ac.uk
> Skype: geraint.pratten
> 
> School of Mathematical and Physical Sciences
> Pevensey 3 Building
> University of Sussex
> Falmer Campus
> Brighton
> BN1 9QH
> United Kingdom

-- 
Ian Hinder
http://members.aei.mpg.de/ianhin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20151005/38e81d0b/attachment-0001.html