<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><br><div><div>On 30 Sep 2016, at 22:35, dumsani <<a href="mailto:g14n8326@campus.ru.ac.za">g14n8326@campus.ru.ac.za</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta content="text/html; charset=windows-1252" http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF">
Hi Roland, Ian, Everyone,<br>
<br>
Thanks for your input. May I add a few points though to elaborate on
my question. Let's assume we run a simulation with 2 MPI processes
(ranks) with each process writing to it's HDF5 own file. Further
assume we have output from 2 restarts on top of the first output,
i.e. we have Simfactory output directories output-0000, output-0001
and output-0002. So, for a variable rho, for instance, we would then
have<br>
output-0000/<parfile>/rho.file_0.h5<br>
output-0000/<parfile>/rho.file_1.h5<br>
<br>
output-0001/<parfile>/rho.file_0.h5<br>
output-0001/<parfile>/rho.file_1.h5<br>
<br>
output-0002/<parfile>/rho.file_0.h5<br>
output-0002/<parfile>/rho.file_1.h5 <br>
<br>
Now, for purposes of visualizing the data, for instance, one would
be interested in collecting all these data chunks into a single
file. The question is how to bundle these into a single file. I
wanted to know if any of the available Carpet tools can help with
this. <br>
<br>
I stand to be corrected, but I seem to recall that some of the
non-ET tools (e.g. SimulationTools) seem to have an abstraction
mechanism for transparently combining CarpetIOHDF5 output from
various checkpoints. Is that the direction I may have to go?<br></div></blockquote><div><br></div><div>Which tool do you want to use? VisIt automatically combines the different components (i.e rectangular regions) from a single run when it loads them. As far as I know, it doesn't understand that you may have the run split into multiple restarts, so I don't know what people do in that case.</div><div><br></div><div>If you want to write your own tool, I think it would be best to load the original raw data in the format that it is written, rather than requiring an additional postprocessing step. In general, the data might be very large, and you may not have the space or time to hold two copies (original and post-processed). There are workarounds; e.g. you can create a single HDF5 file containing HDF5 symbolic links to the other files. I think Roland has done this in the past.</div><div><br></div><div>Erik has been talking about a new file format and library for writing and reading simulation data, which presumably would also hide these details, but I don't think it's ready for production use yet.</div><div><br></div><div>Yes, SimulationTools knows about both splits, and the user doesn't have to worry about it.</div><div><br></div></div><div apple-content-edited="true">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div>-- </div><div>Ian Hinder</div><div><a href="http://members.aei.mpg.de/ianhin">http://members.aei.mpg.de/ianhin</a></div></div></div></div></div>
</div>
<br></body></html>