#1674: Switch to OpenBLAS --------------------------+------------------------------------------------- Reporter: eschnett | Owner: Type: enhancement | Status: review Priority: major | Milestone: Component: Other | Version: development version Resolution: | Keywords: --------------------------+-------------------------------------------------
Comment (by rhaas):
I'll try to answer each question:
These are my workstation, my Linux laptop, my OSX laptop, three different Linux installations in virtual machines. My undestanding is that the issue of compiling/using OpenBLAS only comes up on this types of machines (ie. personal workstations and laptops) at all, yes? For the XSEDE/SciNet/PRACE/whatnot clusters in simfactory we already provide paths to optimized LAPACK/BLAS versions. For private clusters that someone sets up, there would seem to be enough complexity already that choosing the proper LAPACK/BLAS library should be a minor point, eg. I usually find it much harder to get infiniband to work properly.
Maybe I misunderstood. I was under the impression that the suggestion was to have OpenBLAS ignore any installed LAPACK/BLAS and compile its own code since it is hard to determine if the system installed LAPACK is slow. This would only affect its probe-for-installed-versions mode, not the mode where we specify an non-special CCC_DIR. The current instructions for first users for Ubuntu (https://docs.einsteintoolkit.org/et- docs/Simplified_Tutorial_for_New_Users) already suggest installing libatlas-base-dev and the ubuntu option list in simfactory does the same in its comment headers. I vanilla Ubuntu install will also work since it does not install any lapack/blas by default.
I keep at least three different Cactus trees on my workstation at all times: ET_master, ET_release, Zelmani. Each with different sources for the thorns. I have ~40 Cactus trees on my workstation, most of them historical only but maybe an additional to the three listed above that I actually use. I have multiple configurations in Zelmani, one that is mostly vanilla, one for MHD, one for the inversion-symmetry-preserving setup, one that just compiles external libraries for use by other toolkit, several that are -debug variants of the above, several that contain the state of the code as used by other group members. On top of that I have maybe 5-10 Cactus tree on various machines that are builds of Formaline tarballs when I wanted/needed to exactly reproduce behaviour of one of mine/a group member's runs; those probably don't count since they really would actualy *want* to compile everything from scratch to make sure I use the very same code as the last time around and the machines are often clusters in simfactory.
The trees on my workstation usually don't use simfactory to build but use the a variant of debian.cfg to build, the OSX laptop uses osx-yosemite- homebrew-gcc.cfg to compile. No workstation/laptop of mine starts runs using simfactory since they are testing machines only and I find using a script of my own to call mpirun (and log output) more convenient.
My builds usually use simfactory option lists, ET/Zelmani thornlists but that do not use simfactory to build or run. I have not admit I am not sure how using simfactory would change this, unless we'd want to use ENABLE/DISABLE thorn statements in the localhost.ini ini files that sim setup creates when a private workstation is set up.
This may not be the most efficient setup and differ considerably from what others are using. I am also not suggesting this to be the best option, but it is the one that I ended up using.
My main concerns are that somebody who wants to "just try" the ET should not have to spend a long time compiling it, and (just for my own, though I can naturally change options etc to achieve this in any way) I like software to have simple, predictable behaviour that is the same among similar software (ie all ExternalLibraries should behave the same).