Here is an overview of our goal:
We are trying to visualize vectors in the default build of VisIt 2.6.0. Our data is in 3 sets of HDF5 files, one for each of the x, y, and z components. And for each component, there are 48 files (chunked as the output of each of the 48 processors). The simulation uses Cactus AMR grids. The total size of all of these files is roughly 30 GB, and each one individually is roughly 210 MB. We find it either takes an absurd amount of time or doesn't work at all.
We have found one method of visualizing these vectors, but it is outrageously slow.
Working method: This method uses 100% of one CPU and ~57% of the RAM (of 8 GB in total). It takes nearly 25 minutes to render one timestep when displaying just the Vectors module.
Here's what the method includes:
Database correlation between the three components (Correlation Method: Time)
Data-Level comparisons: Between meshes in two different databases *Position*-Based CMFE Simply place it on the target mesh
We modified the expressions to place angled brackets around <Carpet AMR grid>
Made an expression that puts these three components into a vector ex {<MHD_EVOLVE--mhd_st_x>,By_position,Bz_position}
Almost-working method: The only change made to the above process is in the Data-Level Comparisons: (Connectivity based instead of position based) Between meshes in two different databases *Connectivity*-Based CMFE Simply place it on the target mesh
This runs significantly more quickly on the first timestep in the data. But when we try to advance to the next time step in the database correlations, we get this error: ERROR: Vector: ()
viewer: The databases cannot be compared because they have a different number of domains.
We believe this to be a result of the moving AMR boxes, but we cannot seem to work around this. The components of the vector should each have the exact same positions as each other at all times, but VisIt seems to disagree with this.
Our thoughts to resolve this primarily center around finding some manner to make the Connectivity based CMFE function. If that's not possible, we can try and see if building a parallel version of Visit 2.6.0 might help speed this up. Any advise on where to go from here?