<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>I don't think this is by default per say.  I use batchtools
      (instead of simfactory) exactly for this reason and discourage new
      users from restarting in the same directory as the parent
      checkpoints to avoid this exact outcome.  An additional issue that
      can arise is if a job is terminated before the walltime such that
      the data stored in ASCII/HDF5 goes beyond the last checkpoint are
      potential ingestion issues due to data mismatching from the
      restart.  I think overall Kuibit handles this well, but it is has
      been an issue in the past for some users before learning to
      separate restarts into individual directories.</p>
    <p>Cheers,</p>
    <p>Samuel</p>
    <div class="moz-cite-prefix">On 12/16/22 10:01 AM, Bruno Giacomazzo
      wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CADjWfoik1XMwcrx449=0VvsLWhvQb2H=4MX1SdWwdrtdqYsYnw@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <div dir="ltr">
        <div class="gmail_quote">
          <blockquote class="gmail_quote" style="margin:0px 0px 0px
            0.8ex;border-left:1px solid
            rgb(204,204,204);padding-left:1ex">
            <div>
              <div><span style="font-family:&quot;Andale Mono&quot;">-
                  Safety feature to avoid HDF5 files from being
                  corrupted</span><font face="Andale Mono"><span
                    style="font-style:normal"><br>
                  </span></font></div>
              <div><font face="Andale Mono"><span
                    style="font-style:normal">  * Leo requests a feature
                    that would allow the user to e.g., generate one</span></font></div>
              <div><font face="Andale Mono"><span
                    style="font-style:normal">    output file per
                    restart. With kuibit, there was interested in
                    switching from</span></font></div>
              <div><font face="Andale Mono"><span
                    style="font-style:normal">    the ASCII data files
                    to the HDF5 in our research group. However, in a
                    recent</span></font></div>
              <div><font face="Andale Mono"><span
                    style="font-style:normal">    simulation it turned
                    out that a node failure caused a crash as one of the</span></font></div>
              <div><font face="Andale Mono"><span
                    style="font-style:normal">    HDF5 was being written
                    to and we lost all data for an important</span></font></div>
              <div><font face="Andale Mono"><span
                    style="font-style:normal">    gridfunction. If one
                    HDF5 file was written per restart (or another safety</span></font></div>
              <div><font face="Andale Mono"><span
                    style="font-style:normal">    feature was in place),
                    then this would have not been an issue, as only one</span></font></div>
              <div><font face="Andale Mono"><span
                    style="font-style:normal">    of the chunks of data
                    would have been corrupted. Leo will open a ticket</span></font></div>
              <div><font face="Andale Mono"><span
                    style="font-style:normal">    about this.</span></font></div>
            </div>
          </blockquote>
          <div><br>
          </div>
          <div>Isn't this done automatically when using simfactory? I
            have my hdf5 data written in the separate output-00??
            directories (the ones generated by symfactory at each
            restart) so that if one run has problems I do not lose all
            the data.</div>
          <div><br>
          </div>
          <div>Cheers,</div>
          <div>Bruno</div>
          <div><br>
          </div>
        </div>
        <div><br>
        </div>
        -- <br>
        <div dir="ltr" class="gmail_signature">
          <div dir="ltr">
            <div>
              <div dir="ltr">
                <div>
                  <div dir="ltr">
                    <div dir="ltr">
                      <p><font color="#000000">Pr<font face="arial,
                            sans-serif">of. Bruno Giacomazzo<br>
                            Department of Physics<br>
                            University of Milano-Bicocca<br>
                          </font></font><span style="color:rgb(0,0,0)"><font
                            face="arial, sans-serif">Piazza della
                            Scienza 3<br>
                          </font></span><span style="color:rgb(0,0,0)"><font
                            face="arial, sans-serif">20126 Milano<br>
                          </font></span><span
                          style="font-size:12.8px;color:rgb(0,0,0)">Italy</span></p>
                      <p><font color="#000000"><span
                            style="font-size:12.8px">email: </span><span
                            style="font-size:12.8px"><a
                              href="mailto:bruno.giacomazzo@unimib.it"
                              target="_blank" moz-do-not-send="true"
                              class="moz-txt-link-freetext">bruno.giacomazzo@unimib.it</a><br>
                          </span><span style="font-size:12.8px">phone:
                            (+39) 02 6448 2321</span></font><br>
                        <font color="#000000"><span
                            style="font-size:12.8px">web: </span></font><span
                          style="color:rgb(0,0,0);font-size:12.8px"><a
                            href="http://www.brunogiacomazzo.org/"
                            style="font-size:12.8px" target="_blank"
                            moz-do-not-send="true"
                            class="moz-txt-link-freetext">http://www.brunogiacomazzo.org</a></span></p>
                      <p><font color="#000000">---------------------------------------------------------------------<br>
                          <span style="font-size:12.8px">There are only
                            10 types of people in the world:<br>
                          </span><span style="font-size:12.8px">Those
                            who understand binary, and those who don't<br>
                          </span><span style="font-size:12.8px">----------------------------------------------------------------------<br>
                          </span></font></p>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
      <br>
      <fieldset class="moz-mime-attachment-header"></fieldset>
      <pre class="moz-quote-pre" wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@einsteintoolkit.org">Users@einsteintoolkit.org</a>
<a class="moz-txt-link-freetext" href="http://lists.einsteintoolkit.org/mailman/listinfo/users">http://lists.einsteintoolkit.org/mailman/listinfo/users</a>
</pre>
    </blockquote>
  </body>
</html>