[ET Trac] [Einstein Toolkit] #1674: Switch to OpenBLAS

[Einstein Toolkit]

#1674: Switch to OpenBLAS
--------------------------+-------------------------------------------------
  Reporter:  eschnett     |       Owner:                     
      Type:  enhancement  |      Status:  review             
  Priority:  major        |   Milestone:                     
 Component:  Other        |     Version:  development version
Resolution:               |    Keywords:                     
--------------------------+-------------------------------------------------

Comment (by rhaas):

 I'll try to answer each question:

 These are my workstation, my Linux laptop, my OSX laptop, three different
 Linux installations in virtual machines. My undestanding is that the issue
 of compiling/using OpenBLAS only comes up on this types of machines (ie.
 personal workstations and laptops) at all, yes? For the
 XSEDE/SciNet/PRACE/whatnot clusters in simfactory we already provide paths
 to optimized LAPACK/BLAS versions. For private clusters that someone sets
 up, there would seem to be enough complexity already that choosing the
 proper LAPACK/BLAS library should be a minor point, eg. I usually find it
 much harder to get infiniband to work properly.

 Maybe I misunderstood. I was under the impression that the suggestion was
 to have OpenBLAS ignore any installed LAPACK/BLAS and compile its own code
 since it is hard to determine if the system installed LAPACK is slow. This
 would only affect its probe-for-installed-versions mode, not the mode
 where we specify an non-special CCC_DIR. The current instructions for
 first users for Ubuntu (https://docs.einsteintoolkit.org/et-
 docs/Simplified_Tutorial_for_New_Users) already suggest installing
 libatlas-base-dev and the ubuntu option list in simfactory does the same
 in its comment headers. I vanilla Ubuntu install will also work since it
 does not install any lapack/blas by default.

 I keep at least three different Cactus trees on my workstation at all
 times: ET_master, ET_release, Zelmani. Each with different sources for the
 thorns. I have ~40 Cactus trees on my workstation, most of them historical
 only but maybe an additional to the three listed above that I actually
 use. I have multiple configurations in Zelmani, one that is mostly
 vanilla, one for MHD, one for the inversion-symmetry-preserving setup, one
 that just compiles external libraries for use by other toolkit, several
 that are -debug variants of the above, several that contain the state of
 the code as used by other group members. On top of that I have maybe 5-10
 Cactus tree on various machines that are builds of Formaline tarballs when
 I wanted/needed to exactly reproduce behaviour of one of mine/a group
 member's runs; those probably don't count since they really would actualy
 *want* to compile everything from scratch to make sure I use the very same
 code as the last time around and the machines are often clusters in
 simfactory.

 The trees on my workstation usually don't use simfactory to build but use
 the a variant of debian.cfg to build, the OSX laptop uses osx-yosemite-
 homebrew-gcc.cfg to compile. No workstation/laptop of mine starts runs
 using simfactory since they are testing machines only and I find using a
 script of my own to call mpirun (and log output) more convenient.

 My builds usually use simfactory option lists, ET/Zelmani thornlists but
 that do not use simfactory to build or run. I have not admit I am not sure
 how using simfactory would change this, unless we'd want to use
 ENABLE/DISABLE thorn statements in the localhost.ini ini files that sim
 setup creates when a private workstation is set up.

 This may not be the most efficient setup and differ considerably from what
 others are using. I am also not suggesting this to be the best option, but
 it is the one that I ended up using.

 My main concerns are that somebody who wants to "just try" the ET should
 not have to spend a long time compiling it, and (just for my own, though I
 can naturally change options etc to achieve this in any way) I like
 software to have simple, predictable behaviour that is the same among
 similar software (ie all ExternalLibraries should behave the same).

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1674#comment:16>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit