#2059: SimFactory should detect the number of cores automatically --------------------------+------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: review Priority: minor | Milestone: Component: SimFactory | Version: development version Resolution: | Keywords: --------------------------+-------------------------------------------------
Comment (by rhaas):
True, a very point. I attach output files to back up the stated speed improvement for 12 MPI ranks and 1 threads vs. 2 MPI ranks and 6 threads on my workstation. Speed difference is (for static_tov.par) 200Msun/hr vs. 250Msun/hr so ~25% speedup. I also tested 12 threads (hyperthreading) which (keeping the parfile unchanged) makes it much slower: 10M/hr.
Question is then: do we want this? Ie having simfactory's auto-detect logic set up an "unusual" setting that differs from what is typically used on clusters? Mostly interesting if (which I assume will happen) to set up a new cluster, people will start from "sim setup" on the login node. Typically I am not concerned with speed on individual workstations, reproducibility and debug-ability are much more important to me on my workstation (since I used it to debug and test code but not to run simulations).
Testsuites may be a issue (namely that we want both multiple MPI ranks and multiple threads in a testsuite run) but those always can be enforced by passing --num-threads and --procs options to simfactory.
So MPI only gives more speed (very nice). It is less "common" on clusters which may be an issue if "sim setup" is used as a base for cluster installations (it will have to be modified anyway).
I tended to prefer having something closer to a cluster setup, but really see the advantages of having faster evolutions so could certainly live with it. In particular for actual laptops/workstations only. Is there anyone who cannot live with either option?