On Wed, Sep 02, 2015 at 10:50:53AM +0200, Ian Hinder wrote:
I have thought about this a fair amount before. Many thorns use their own output routines because Cactus does not provide what they need. The Cactus output methods are designed to output grid variables and grid arrays, but many thorns would like to output tables of data which do not correspond to Cactus variables.
What I was thinking about wasn't grid functions or arrays, but simple scalars (or lists thereof). Cactus does provide output for those, just not in HDF5.
For example, AHFinderDirect and Multipole. PunctureTracker handles this by creating Cactus variables for its data. The down-side to this is that all the "trackers" are always output, leading to a very large number of columns in the ascii output, making it impossible for a human to understand without tedious column-counting.
I agree that outputting invalid data is wasteful. The same happens for spherical surfaces.
Multipole cannot use this mechanism, because it wants to write ascii files with different names for different l,m,r,var combinations.
Indeed. But maybe that's not a good choice to begin with. I for my part would rather have all data in one file. It would make such an ASCII file unreadable, but I rarely open them in an editor anyway.
HDF5 output of grid arrays is very immature, and rarely does what you want (e.g. it usually creates one dataset per iteration, making reading the file very tedious).
I am not sure about all grid arrays (simply didn't think hard enough about it), but something like scalar variables and groups thereof should be 'simple' enough to be output as time series, as one scalar series per dataset, grouped inside hdf5.
Instead, I think there should be a Cactus mechanism for outputting "tables" of data to "files". These tables would have named "columns", and the "files" could be real ASCII files, or extensible datasets in HDF5 files. The "rows" of the tables would correspond to output iterations.
Up to here this is scalar output as we have it.
The data would not have to correspond to Cactus variables.
I think it would be better to make them such. The output routines would need to be a bit cleverer when it comes to their output.
Output would happen via aliased function calls. Probably we would have an initial call to "register" the output table, and subsequent calls to add to it.
So far the workflow was different. You don't register variables to be output. You register variables, and the output thorns decide what gets output (using parameters). This seems cleaner to me, better separated. If this mechanism is missing something, we could think about adding it. What I can think of immediately are two things:
- provide a way to specify 'column names' for each variable This could either be done via tags, and/or via a new CCTK function if we think we need to change this at runtime - provide a way to specify validity of a variable - such that invalid data isn't output (by default) Designing this might be a little harder. We should avoid extra calls just for this. One possibility would be an extra "valid" state for each variable. But that doubles the number of variables and wouldn't work for arrays anyway where only part of it might be valid, and part is not. Leaving arrays aside for a moment, and assuming we would leave the default of this 'state' to 'valid', we could provide a CCTK call for thorns to specify that certain variables do not contain valid data at a given time step. This would be collected as list (of variable ids), and can be used by output thorns.
Frank