Hello,
I recently got ET working on my personal computer (virtual machine with ubuntu using the default ubuntu .cfg file), but I am having issues compiling the code on a local computer cluster using a custom .cfg file based off of the ubuntu .cfg (see end of email). However, when I attempt to compile the code on the cluster, it gets to the point where it begins building the executable, but then it prints out an error as seen below.
**Start Code Creating cactus_sim in /n/a3/srosofsky/Cactus/exe from [list of files]
/n/a3/srosofsky/Cactus/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topolog\ y-linux.o): In function `hwloc_linux_set_area_membind': /n/a3/srosofsky/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology\ -linux.c:1224: undefined reference to `mbind' /n/a3/srosofsky/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology\ -linux.c:1243: undefined reference to `mbind' /n/a3/srosofsky/Cactus/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topolog\ y-linux.o): In function `hwloc_linux_find_kernel_max_numnodes': /n/a3/srosofsky/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology\ -linux.c:1347: undefined reference to `get_mempolicy' /n/a3/srosofsky/Cactus/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topolog\ y-linux.o): In function `hwloc_linux_get_area_membind': /n/a3/srosofsky/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology\ -linux.c:1445: undefined reference to `get_mempolicy' /n/a3/srosofsky/Cactus/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topolog\ y-linux.o): In function `hwloc_linux_get_thisthread_membind': /n/a3/srosofsky/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology\ -linux.c:1392: undefined reference to `get_mempolicy' /n/a3/srosofsky/Cactus/configs/sim/scratch/external/hwloc/lib/libhwloc.a(topolog\ y-linux.o): In function `hwloc_linux_set_thisthread_membind': /n/a3/srosofsky/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology\ -linux.c:1291: undefined reference to `set_mempolicy' /n/a3/srosofsky/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology\ -linux.c:1302: undefined reference to `migrate_pages' /n/a3/srosofsky/Cactus/configs/sim/scratch/build/hwloc/hwloc-1.10.1/src/topology\ -linux.c:1314: undefined reference to `set_mempolicy' collect2: ld returned 1 exit status make[1]: *** [/n/a3/srosofsky/Cactus/exe/cactus_sim] Error 1 make: *** [sim] Error 2 **End code
The above error occurs when the MPI_DIR in the .cfg file is set to /usr/lib64/openmpi/. I should note that when I change MPI_DIR to BUILD I get a different error message, but still at the same point in the compilation process.
**Start Code
Creating cactus_sim in /n/a3/srosofsky/Cactus/exe from [file list]
/usr/bin/ld: cannot find -lmpi_usempif08 collect2: ld returned 1 exit status make[1]: *** [/n/a3/srosofsky/Cactus/exe/cactus_sim] Error 1 make: *** [sim] Error 2
**End Code
Also, I looked at tickets 1717 and 1764 for possible overlap on this issue, but I had difficulty following how the issues in said tickets were resolved. Furthermore, the libnuma-dev and libltdl-dev libraries were installed but the problem still persists. I should note that I am currently using ET version ET_2015_11.
Any suggestions would be greatly appreciated.
Best, Shawn
Here is the configuration file that I used.
**Start Code
#MPI_DIR=BUILD
MPI_DIR = /usr/lib64/openmpi/
PTHREADS_DIR = NO_BUILD
HDF5_DIR = BUILD
HWLOC_DIR = BUILD
HWLOC_EXTRA_LIBS = hwloc
**End Code
On Sun, Feb 14, 2016 at 11:27:48PM -0500, Shawn Rosofsky wrote:
HWLOC_EXTRA_LIBS = hwloc
Try adding numa here. The missing symbols should be defined in libnuma, so something is using it, but not pulling this dependency in. My guess is it's the hwloc thorn. It is supposed to detect this automatically, but something might go wrong. Looking at the code it seems to only do this check for an already installed version, not for the self-built you are using.
Also: HWLOC_EXTRA_LIBS shouldn't really be set to 'hwloc'. That's the hwloc library itself and should not need to be added. Replace 'hwloc' with 'numa'.
Frank
It works now. Thanks for the help.
On Sun, Feb 14, 2016 at 11:38 PM, Frank Loeffler knarf@cct.lsu.edu wrote:
On Sun, Feb 14, 2016 at 11:27:48PM -0500, Shawn Rosofsky wrote:
HWLOC_EXTRA_LIBS = hwloc
Try adding numa here. The missing symbols should be defined in libnuma, so something is using it, but not pulling this dependency in. My guess is it's the hwloc thorn. It is supposed to detect this automatically, but something might go wrong. Looking at the code it seems to only do this check for an already installed version, not for the self-built you are using.
Also: HWLOC_EXTRA_LIBS shouldn't really be set to 'hwloc'. That's the hwloc library itself and should not need to be added. Replace 'hwloc' with 'numa'.
Frank
users@lists.einsteintoolkit.org
-
Frank Loeffler -
Shawn Rosofsky