[ET Trac] [Einstein Toolkit] #1828: Script to collect AH shape information into one HDF5 file

[Einstein Toolkit]

#1828: Script to collect AH shape information into one HDF5 file
--------------------------+-------------------------------------------------
  Reporter:  knarf        |       Owner:                     
      Type:  enhancement  |      Status:  reopened           
  Priority:  optional     |   Milestone:                     
 Component:  Cactus       |     Version:  development version
Resolution:               |    Keywords:                     
--------------------------+-------------------------------------------------

Comment (by knarf):

 Thanks for your reviews. There are obviously many ways to improve this
 script. Some I implemented now, some others are invited to.

 Replying to [comment:2 rhaas]:
 > * the name should not contain "visit" since xdmf files are also readable
 by other tools eg Paraview, instead maybe "CollectAHShapesInXdmf.py"

 You are right. Maybe just AH2xdmf.py?

 > * why is there a need to specify an HDF5 file? I would sometimes like to
 just visualize the horizon without having any hdf5 output present (eg for
 binary black hole simulations, see eg
 https://www.youtube.com/watch?v=ZsODZW0VuhQ)

 Again, you are right. In the case I needed it for, I had to align time
 steps to existing hdf5 files. That argument is optional now, and if not
 given all timesteps in AHs are used.

 > * the script makes assumptions on the name of the files which are not
 always true since the AHFinderDirect parameter {{{h_base_file_name}}} lets
 you change the "h" to something else (see comment in param.ccl). It should
 document these assumptions more prominently rather than only in the middle
 part of the source code.

 It now is part of the help text. It would be difficult to find files with
 non-default names though, unless one were to add an option for the regex
 which matches those files, and returns the horizon and timestep numbers.
 This is possible, and such a contribution would be welcome.

 > * would it be possible to also support AHFinderDirect's
 {{{output_HDF5_files}}} option?

 I have to confess, I didn't even know about that option. Of course that
 would be most desirable, and maybe then one wouldn't even need to create a
 separate h5 file but could simply use the existing one.
 Since I don't have any of such output right now, however, I will only add
 this later, or someone else is welcome now.

 > * an alternative to having Xdmf+HDF5 may be to store all data inline in
 the Xdmf file (ASCII in that case) if this is not provibitively large. It
 would have the advantage of fewer files having to be carried around

 One could, but I didn't try. A typical horizon shape needs on the order of
 50kB if stored in binary format (obviously depending on N_rho and
 N_theta). hdf5 adds overhead to that. hdf5 compression isn't efficient for
 that size too. I would estimate ASCII to need a small factor (2-3?) more
 than that, which wouldn't make it a size issue. For an example of about
 3400 horizons (19x19), I get an hdf5 file size of about 240MB. Reading
 ASCII is most likely much slower, having to parse the ASCII values, but
 with that size that should also not be a problem. In short: I don't see a
 reason why it shouldn't work, but I didn't see a reason why to go that
 route. I came "from the other side": I started with hdf5 and wanted an xml
 description of the file so a visualization tool can just read it.
 Your argument with the number of files is a good one, but reducing the
 gazillion AH files (for each timestep) to just two per horizon is already
 sufficient for me.

 > * the script should support a {{{--help}}} option explicitly, the help
 text should start with the word "Usage", required argument should not be
 enclosed in brackets but be bare, the
 [[https://en.wikipedia.org/wiki/Usage_message#Pattern|fount of all
 knowledge]] suggests a particular format.

 It now has. Thanks for that. It doesn't use a full-blown option parser
 though (yet).

 > * it would be nice if error message (the usage is no error message) were
 output to stderr rather than stdout

 It now does.

 > * rather than adding dummy meshes, would it not be better to re-use the
 mesh from the last timestep? This would be visually less jarring than
 having an AH vanish

 Not if you have an AH disappear, like you can have in BBH simulations. One
 could add an option for this, which most likely should be per horizon.
 Once the script has found a home everyone is welcome to add that
 possibility.

 > * this is an if(True) statement in there. We should either do as the
 comment above says or remove the "else" branch. Users should not have to
 modify the script code to chose options.

 I agree. I just didn't have the patience to fully implement possible
 options, but had the code for one obvious case already there. It would
 have been waste to just delete it, so I "commented" it that way. At least
 it would be an easy code change.
 Having said that - now that the input hdf5 file is optional, both code
 paths can actually be taken.

 > * compare this to Eloisa's older scripts
 http://svn.cactuscode.org/VizTools/VTKutils/trunk/AHFVTK/ which produce
 VTK files instead (they require fewer tools, only bash and (g)awk).

 I looked into this and even used it once or twice (Thanks Elo!), but if I
 remember correctly, vtk doesn't (easily) support multiple timesteps per
 file (or visit's reader didn't) - and once I had to 'roll my own' anyway I
 felt that python would be a better choice for parsing text files and
 writing data to hdf5.

 Replying to [comment:3 hinder]:
 > How hard would it be to modify AHFinderDirect to output such files
 directly?

 It shouldn't be very hard. In my case that was not an option. I already
 had the data and didn't want to re-run. Also, things would get complicated
 when multiple restarts come into the game.

 > Could it be done already in a parameter file by storing the AH shape in
 a spherical surface and then outputting the spherical surface using
 CarpetIOHDF5?

 Outputting the spherical surface would be a good option (also because
 other thorns would benefit from that too). I was thinking about that but
 then found the AH files easy to parse and the fastest route to what I
 needed. A similar script for SphericalSurface output shouldn't be that
 complicated.

 > Is there a place that such scripts are all listed?

 We do have a 'tools'/'utilities' repository, but in that case I thought
 about putting it into the AHFinderDirect repository, since it is obviously
 only reading those output files and would be useless without.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1828#comment:5>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit