#1828: Script to collect AH shape information into one HDF5 file --------------------------+------------------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: review Priority: optional | Milestone: Component: Cactus | Version: development version Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by knarf):

* status: new => review

#1828: Script to collect AH shape information into one HDF5 file --------------------------+------------------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: review Priority: optional | Milestone: Component: Cactus | Version: development version Resolution: | Keywords: --------------------------+-------------------------------------------------

Comment (by hinder):

Thanks for the script!

How hard would it be to modify AHFinderDirect to output such files directly? Could it be done already in a parameter file by storing the AH shape in a spherical surface and then outputting the spherical surface using CarpetIOHDF5? This information would be quite "raw". AHFinderDirect could then maybe output a corresponding XMDF file so it would be easy to open in a visualisation tool.

I agree with Roland that it would be really good if the script could be used without a reference HDF5 file. For example, maybe you could provide a list of times or iterations in an argument, or another file, or in standard input. Or maybe just arguments for --starttime, --endtime, --deltatime. Or the corresponding iteration equivalent.

Is there a place that such scripts are all listed?

#1828: Script to collect AH shape information into one HDF5 file --------------------------+------------------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: reopened Priority: optional | Milestone: Component: Cactus | Version: development version Resolution: | Keywords: --------------------------+-------------------------------------------------

Comment (by knarf):

Thanks for your reviews. There are obviously many ways to improve this script. Some I implemented now, some others are invited to.

Replying to [comment:2 rhaas]:

• the name should not contain "visit" since xdmf files are also readable

by other tools eg Paraview, instead maybe "CollectAHShapesInXdmf.py"

You are right. Maybe just AH2xdmf.py?

• why is there a need to specify an HDF5 file? I would sometimes like to

just visualize the horizon without having any hdf5 output present (eg for binary black hole simulations, see eg https://www.youtube.com/watch?v=ZsODZW0VuhQ)

Again, you are right. In the case I needed it for, I had to align time steps to existing hdf5 files. That argument is optional now, and if not given all timesteps in AHs are used.

• the script makes assumptions on the name of the files which are not

always true since the AHFinderDirect parameter {{{h_base_file_name}}} lets you change the "h" to something else (see comment in param.ccl). It should document these assumptions more prominently rather than only in the middle part of the source code.

It now is part of the help text. It would be difficult to find files with non-default names though, unless one were to add an option for the regex which matches those files, and returns the horizon and timestep numbers. This is possible, and such a contribution would be welcome.

• would it be possible to also support AHFinderDirect's

{{{output_HDF5_files}}} option?

I have to confess, I didn't even know about that option. Of course that would be most desirable, and maybe then one wouldn't even need to create a separate h5 file but could simply use the existing one. Since I don't have any of such output right now, however, I will only add this later, or someone else is welcome now.

• an alternative to having Xdmf+HDF5 may be to store all data inline in

the Xdmf file (ASCII in that case) if this is not provibitively large. It would have the advantage of fewer files having to be carried around

One could, but I didn't try. A typical horizon shape needs on the order of 50kB if stored in binary format (obviously depending on N_rho and N_theta). hdf5 adds overhead to that. hdf5 compression isn't efficient for that size too. I would estimate ASCII to need a small factor (2-3?) more than that, which wouldn't make it a size issue. For an example of about 3400 horizons (19x19), I get an hdf5 file size of about 240MB. Reading ASCII is most likely much slower, having to parse the ASCII values, but with that size that should also not be a problem. In short: I don't see a reason why it shouldn't work, but I didn't see a reason why to go that route. I came "from the other side": I started with hdf5 and wanted an xml description of the file so a visualization tool can just read it. Your argument with the number of files is a good one, but reducing the gazillion AH files (for each timestep) to just two per horizon is already sufficient for me.

• the script should support a {{{--help}}} option explicitly, the help

text should start with the word "Usage", required argument should not be enclosed in brackets but be bare, the [[https://en.wikipedia.org/wiki/Usage_message#Pattern%7Cfount of all knowledge]] suggests a particular format.

It now has. Thanks for that. It doesn't use a full-blown option parser though (yet).

• it would be nice if error message (the usage is no error message) were

output to stderr rather than stdout

It now does.

• rather than adding dummy meshes, would it not be better to re-use the

mesh from the last timestep? This would be visually less jarring than having an AH vanish

Not if you have an AH disappear, like you can have in BBH simulations. One could add an option for this, which most likely should be per horizon. Once the script has found a home everyone is welcome to add that possibility.

• this is an if(True) statement in there. We should either do as the

comment above says or remove the "else" branch. Users should not have to modify the script code to chose options.

I agree. I just didn't have the patience to fully implement possible options, but had the code for one obvious case already there. It would have been waste to just delete it, so I "commented" it that way. At least it would be an easy code change. Having said that - now that the input hdf5 file is optional, both code paths can actually be taken.

• compare this to Eloisa's older scripts

http://svn.cactuscode.org/VizTools/VTKutils/trunk/AHFVTK/ which produce VTK files instead (they require fewer tools, only bash and (g)awk).

I looked into this and even used it once or twice (Thanks Elo!), but if I remember correctly, vtk doesn't (easily) support multiple timesteps per file (or visit's reader didn't) - and once I had to 'roll my own' anyway I felt that python would be a better choice for parsing text files and writing data to hdf5.

Replying to [comment:3 hinder]:

How hard would it be to modify AHFinderDirect to output such files

directly?

It shouldn't be very hard. In my case that was not an option. I already had the data and didn't want to re-run. Also, things would get complicated when multiple restarts come into the game.

Could it be done already in a parameter file by storing the AH shape in

a spherical surface and then outputting the spherical surface using CarpetIOHDF5?

Outputting the spherical surface would be a good option (also because other thorns would benefit from that too). I was thinking about that but then found the AH files easy to parse and the fastest route to what I needed. A similar script for SphericalSurface output shouldn't be that complicated.

Is there a place that such scripts are all listed?

We do have a 'tools'/'utilities' repository, but in that case I thought about putting it into the AHFinderDirect repository, since it is obviously only reading those output files and would be useless without.

#1828: Script to collect AH shape information into one HDF5 file --------------------------+------------------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: review Priority: optional | Milestone: ET_2015_11 Component: Cactus | Version: development version Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by knarf):

* milestone: => ET_2015_11