<div dir="ltr">When using simfactory, if I write in my parameter file that the output directory for the hdf5 files is "./hdf5" (e.g., CarpetIOHDF5::out2D_dir = "./hdf5_2D") then this directory is created in the output-00?? directory. Therefore each run creates a separate hdf5 directory in the corresponding output-00?? one. This is how I avoid losing all data if one run has problems.<div><br></div><div>Cheers,</div><div>Bruno</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il giorno ven 16 dic 2022 alle ore 10:10 Samuel Tootle <<a href="mailto:tootle@itp.uni-frankfurt.de">tootle@itp.uni-frankfurt.de</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>I don't think this is by default per say. I use batchtools
(instead of simfactory) exactly for this reason and discourage new
users from restarting in the same directory as the parent
checkpoints to avoid this exact outcome. An additional issue that
can arise is if a job is terminated before the walltime such that
the data stored in ASCII/HDF5 goes beyond the last checkpoint are
potential ingestion issues due to data mismatching from the
restart. I think overall Kuibit handles this well, but it is has
been an issue in the past for some users before learning to
separate restarts into individual directories.</p>
<p>Cheers,</p>
<p>Samuel</p>
<div>On 12/16/22 10:01 AM, Bruno Giacomazzo
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div><span style="font-family:"Andale Mono"">-
Safety feature to avoid HDF5 files from being
corrupted</span><font face="Andale Mono"><span style="font-style:normal"><br>
</span></font></div>
<div><font face="Andale Mono"><span style="font-style:normal"> * Leo requests a feature
that would allow the user to e.g., generate one</span></font></div>
<div><font face="Andale Mono"><span style="font-style:normal"> output file per
restart. With kuibit, there was interested in
switching from</span></font></div>
<div><font face="Andale Mono"><span style="font-style:normal"> the ASCII data files
to the HDF5 in our research group. However, in a
recent</span></font></div>
<div><font face="Andale Mono"><span style="font-style:normal"> simulation it turned
out that a node failure caused a crash as one of the</span></font></div>
<div><font face="Andale Mono"><span style="font-style:normal"> HDF5 was being written
to and we lost all data for an important</span></font></div>
<div><font face="Andale Mono"><span style="font-style:normal"> gridfunction. If one
HDF5 file was written per restart (or another safety</span></font></div>
<div><font face="Andale Mono"><span style="font-style:normal"> feature was in place),
then this would have not been an issue, as only one</span></font></div>
<div><font face="Andale Mono"><span style="font-style:normal"> of the chunks of data
would have been corrupted. Leo will open a ticket</span></font></div>
<div><font face="Andale Mono"><span style="font-style:normal"> about this.</span></font></div>
</div>
</blockquote>
<div><br>
</div>
<div>Isn't this done automatically when using simfactory? I
have my hdf5 data written in the separate output-00??
directories (the ones generated by symfactory at each
restart) so that if one run has problems I do not lose all
the data.</div>
<div><br>
</div>
<div>Cheers,</div>
<div>Bruno</div>
<div><br>
</div>
</div>
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">
<p><font color="#000000">Pr<font face="arial,
sans-serif">of. Bruno Giacomazzo<br>
Department of Physics<br>
University of Milano-Bicocca<br>
</font></font><span style="color:rgb(0,0,0)"><font face="arial, sans-serif">Piazza della
Scienza 3<br>
</font></span><span style="color:rgb(0,0,0)"><font face="arial, sans-serif">20126 Milano<br>
</font></span><span style="font-size:12.8px;color:rgb(0,0,0)">Italy</span></p>
<p><font color="#000000"><span style="font-size:12.8px">email: </span><span style="font-size:12.8px"><a href="mailto:bruno.giacomazzo@unimib.it" target="_blank">bruno.giacomazzo@unimib.it</a><br>
</span><span style="font-size:12.8px">phone:
(+39) 02 6448 2321</span></font><br>
<font color="#000000"><span style="font-size:12.8px">web: </span></font><span style="color:rgb(0,0,0);font-size:12.8px"><a href="http://www.brunogiacomazzo.org/" style="font-size:12.8px" target="_blank">http://www.brunogiacomazzo.org</a></span></p>
<p><font color="#000000">---------------------------------------------------------------------<br>
<span style="font-size:12.8px">There are only
10 types of people in the world:<br>
</span><span style="font-size:12.8px">Those
who understand binary, and those who don't<br>
</span><span style="font-size:12.8px">----------------------------------------------------------------------<br>
</span></font></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Users mailing list
<a href="mailto:Users@einsteintoolkit.org" target="_blank">Users@einsteintoolkit.org</a>
<a href="http://lists.einsteintoolkit.org/mailman/listinfo/users" target="_blank">http://lists.einsteintoolkit.org/mailman/listinfo/users</a>
</pre>
</blockquote>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><p><font color="#000000">Pr<font face="arial, sans-serif">of. Bruno Giacomazzo<br>Department of Physics<br>University of Milano-Bicocca<br></font></font><span style="color:rgb(0,0,0)"><font face="arial, sans-serif">Piazza della Scienza 3<br></font></span><span style="color:rgb(0,0,0)"><font face="arial, sans-serif">20126 Milano<br></font></span><span style="font-size:12.8px;color:rgb(0,0,0)">Italy</span></p><p><font color="#000000"><span style="font-size:12.8px">email: </span><span style="font-size:12.8px"><a href="mailto:bruno.giacomazzo@unimib.it" target="_blank">bruno.giacomazzo@unimib.it</a><br></span><span style="font-size:12.8px">phone: (+39) 02 6448 2321</span></font><br><font color="#000000"><span style="font-size:12.8px">web: </span></font><span style="color:rgb(0,0,0);font-size:12.8px"><a href="http://www.brunogiacomazzo.org/" style="font-size:12.8px" target="_blank">http://www.brunogiacomazzo.org</a></span></p><p><font color="#000000">---------------------------------------------------------------------<br><span style="font-size:12.8px">There are only 10 types of people in the world:<br></span><span style="font-size:12.8px">Those who understand binary, and those who don't<br></span><span style="font-size:12.8px">----------------------------------------------------------------------<br></span></font></p></div></div></div></div></div></div></div>