-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello all,
I run a virtual machine (one of the slaves for Jenkins though I think it is hardly every used) inside of VirtualBox. I allocated 4 cores (and 4GB of RAM) for it. The host is a Intel(R) Core(TM) i7-2600 CPU which supports symmetric multithreading (ht in cpuinfo). hwloc detects this also inside the the virtual machine and thus sets the default number of threads to 4, however other places in the code disagree and I get a warning that more threads than physical SMT units are used:
INFO (hwloc): Extracting CPU/cache/memory properties: There are 1 PUs per core (aka hardware SMT threads) There are 2 threads per core (aka SMT threads used) WARNING: This is larger than the number of hardware SMT threads
VirtualBox apparently does not fully support this and indeed I end up with 8 concurrent threads which make the test unbearably slow (I had thought the ML tests were stuck).
Explicitly setting OMP_NUM_THREADS=2 fixes the problem for me. If possible it would be nice though if hwloc detected this situation and did not start as many threads (useful eg for new users who use the ET virtual machine to try out Cactus).
I attach the log file output.
Yours, Roland
- -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.
On 2013-09-30, at 0:03 , Roland Haas roland.haas@physics.gatech.edu wrote:
Signed PGP part Hello all,
I run a virtual machine (one of the slaves for Jenkins though I think it is hardly every used) inside of VirtualBox. I allocated 4 cores (and 4GB of RAM) for it. The host is a Intel(R) Core(TM) i7-2600 CPU which supports symmetric multithreading (ht in cpuinfo). hwloc detects this also inside the the virtual machine and thus sets the default number of threads to 4, however other places in the code disagree and I get a warning that more threads than physical SMT units are used:
INFO (hwloc): Extracting CPU/cache/memory properties: There are 1 PUs per core (aka hardware SMT threads) There are 2 threads per core (aka SMT threads used) WARNING: This is larger than the number of hardware SMT threads
VirtualBox apparently does not fully support this and indeed I end up with 8 concurrent threads which make the test unbearably slow (I had thought the ML tests were stuck).
Explicitly setting OMP_NUM_THREADS=2 fixes the problem for me. If possible it would be nice though if hwloc detected this situation and did not start as many threads (useful eg for new users who use the ET virtual machine to try out Cactus).
I attach the log file output.
Roland
Unfortunately, hwloc does not choose the number of MPI processes or OpenMP threads. It seems that you are starting 2 MPI processes with 4 threads each, which are 8 threads altogether, and which over-subscribes the cores you allocated to the VM. hwloc then warns about this, which is correct.
The number of threads is chosen via Simfactory. You probably need to update the machine description's default number of OpenMP threads.
It is a good idea to let hwloc choose the number of OpenMP threads. We should introduce an option for this! Please open a feature request.
-erik
Hello Erik, all,
Unfortunately, hwloc does not choose the number of MPI processes or OpenMP threads. It seems that you are starting 2 MPI processes with 4 threads each, which are 8 threads altogether, and which over-subscribes the cores you allocated to the VM. hwloc then warns about this, which is correct.
True.
The number of threads is chosen via Simfactory. You probably need to update the machine description's default number of OpenMP threads.
Well actually, I was not using simfactory to run the test, only to compile the executable. Thinking about this now, I think I was simply not looking at this correctly. Since OMP_NUM_THREADS was not set I got 4 threads (as many as there are cores which is the OpenMP default) per process. However since I was running 2 MPI processes I was obviously oversubscribing. This had nothing to do with SMT or virtual machines. The error was sitting right in front of the keyboard.
It is a good idea to let hwloc choose the number of OpenMP threads. We should introduce an option for this! Please open a feature request.
Oh, I had no intention claiming that hwloc is doing anything wrong. A feature request has been created.
Yours, Roland
users@lists.einsteintoolkit.org
-
Erik Schnetter -
Roland Haas -
Roland Haas