#1465: Boost doesn't honor parallel build options -----------------------------------+---------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Keywords: | -----------------------------------+---------------------------------------- ExternalLibraries/Boost doesn't honor parallel build options. This is because it doesn't use 'make' to build, but a tool called 'b2', which isn't using 'make's environment variables, nor is it capable to interact with the jobserver make provides. Manually parsing MAKEFLAGS inside the configure script of the thorn also doesn't work, as this doesn't contain the requested number of jobs, but only the fact that a parallel build was requested and a pointer to the jobserver. Manually interacting with this job server might be possible for gnu make, but is probably messy and highly likely not portable.
However, building a huge package like Boost in parallel is something we have to achieve. On my workstation it reduces the time used for building Boost alone from about 500s to 60s.
The only alternative that is easy enough to implement that I can see right now is to make it possible to pass a number of processes used for building to Cactus (in addition to passing it to make), and Boost using this variable for the parallel build. Simfactory should then also pass this to Cactus. One downside of this would be that if multiple libraries were to be built in parallel you might end up using almost twice as many jobs than specified. On the other hand, this doesn't seem to be the case right now and even if using twice as many might be better than many times too few.
#1465: Boost doesn't honor parallel build options ------------------------------------+--------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Resolution: | Keywords: ------------------------------------+---------------------------------------
Comment (by knarf):
Building only the necessary part of Boost for the ET would be another alternative, but I wouldn't know if this is possible, nor how. As it is right now, compiling Boost uses 1/4 of the build time of the entire ET (including Boost).
#1465: Boost doesn't honor parallel build options ------------------------------------+--------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Resolution: | Keywords: ------------------------------------+---------------------------------------
Comment (by rhaas):
Boost is not part of the ET yet, is it? It should have been removed from the thornlist again. If it is still in (and not commented out), it should be commented out.
#1465: Boost doesn't honor parallel build options ------------------------------------+--------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Resolution: | Keywords: ------------------------------------+---------------------------------------
Comment (by eschnett):
Boost is in the thorn list, but commented out.
#1465: Boost doesn't honor parallel build options ------------------------------------+--------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Resolution: | Keywords: ------------------------------------+---------------------------------------
Comment (by knarf):
It is in the thornlist, but commented out. Everything mentioned in the thornlist is meant to be part of the ET. If it isn't, then it shouldn't be mentioned there. Regardless of that, since we use Boost for benchmarks this should be addressed.
#1465: Boost doesn't honor parallel build options ------------------------------------+--------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Resolution: | Keywords: ------------------------------------+---------------------------------------
Comment (by eschnett):
It turns out that we don't have access to the make flags (-j N) in this configure script, since we have already been called from inside a parallel make. We do have access to the Unix pipes that make uses to coordinate multiple parallel makes, but this is not useful for Boost's configure system.
I would close this issue, stating that we rather build Boost ahead of time (e.g. via Simfactoy3).
#1465: Boost doesn't honor parallel build options ------------------------------------+--------------------------------------- Reporter: knarf | Owner: Type: enhancement | Status: closed Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Resolution: wontfix | Keywords: ------------------------------------+--------------------------------------- Changes (by knarf):
* status: new => closed * resolution: => wontfix
Comment:
True. And Simfactory would then pass the 'right' option to whatever build system a library uses.
trac@lists.einsteintoolkit.org
-
Einstein Toolkit