#1732: hw fails to link with cuda --------------------+------------------------------------------------------- Reporter: knarf | Owner: Type: defect | Status: new Priority: major | Milestone: Component: Other | Version: development version Keywords: | --------------------+------------------------------------------------------- I get link errors like this: {{{ /home/knarf/psamr/configs/sim/scratch/external/hwloc/lib/libhwloc.a (topology-cuda.o): In function `hwloc_cuda_backend_notify_new_object': /home/knarf/psamr/configs/sim/scratch/build/hwloc/hwloc-1.7.2/src /topology-cuda.c:122: undefined reference to `cudaGetDeviceProperties' /home/knarf/psamr/configs/sim/scratch/external/hwloc/lib/libhwloc.a (topology-cuda.o): In function `hwloc_cuda_query_devices': /home/knarf/psamr/configs/sim/scratch/build/hwloc/hwloc-1.7.2/src /topology-cuda.c:38: undefined reference to `cudaGetDeviceCount' /home/knarf/psamr/configs/sim/scratch/external/hwloc/lib/libhwloc.a (topology-cuda.o): In function `hwloc_cudart_get_device_pci_ids': /home/knarf/psamr/configs/sim/scratch/build/hwloc/hwloc-1.7.2/include/hwloc/cudart.h:49: undefined reference to `cudaGetDeviceProperties' collect2: error: ld returned 1 exit status }}}
hwloc is built (system version present, but deemed too old). The configuration is using the standard debian option list (on spine). Everything is dev-version, clean checkout.
It seems to me that hwloc picks up a cuda dependency (present on system), but the linker isn't told about it (no cuda present in Cactus config).
#1732: hwloc in Cactus fails to link with cuda ---------------------+------------------------------------------------------ Reporter: knarf | Owner: Type: defect | Status: new Priority: major | Milestone: Component: Other | Version: development version Resolution: | Keywords: ---------------------+------------------------------------------------------
#1732: hwloc in Cactus fails to link with cuda ---------------------+------------------------------------------------------ Reporter: knarf | Owner: Type: defect | Status: new Priority: major | Milestone: Component: Other | Version: development version Resolution: | Keywords: ---------------------+------------------------------------------------------
Comment (by eschnett):
Ah yes, the joys of building late... We need to tell Cactus what libraries we want before hwloc is installed, i.e. before its pkgconfig file is written. The standard solutions are either (1) explicitly disable CUDA integration in hwloc, or (2) add these libraries to LDFLAGS and LIBS for this particular machine. Note that setting LIBS overwrites, so you'll need to manually add the automatically detected libraries as well.
#1732: hwloc in Cactus fails to link with cuda ---------------------+------------------------------------------------------ Reporter: knarf | Owner: Type: defect | Status: new Priority: major | Milestone: Component: Other | Version: development version Resolution: | Keywords: ---------------------+------------------------------------------------------
Comment (by knarf):
For this particular case I think that a switch for using Cuda or not would be the easiest thing to do. If using Cuda we would need to point hwloc to the right installation anyway, i.e. the one Cactus plans to use. What do you (and others) think?
The current situation means that the ET does not build if hwloc is built by the ET, and Cuda is installed, but not used by Cactus (which is the default).
#1732: hwloc in Cactus fails to link with cuda ---------------------+------------------------------------------------------ Reporter: knarf | Owner: Type: defect | Status: closed Priority: major | Milestone: Component: Other | Version: development version Resolution: fixed | Keywords: ---------------------+------------------------------------------------------ Changes (by knarf):
* status: new => closed * resolution: => fixed
Comment:
I disabled cuda and opencl discovery in hwloc for now. If it is needed, we would need to check for support in Cactus first, and point hwloc to the right location as well.
#1732: hwloc in Cactus fails to link with cuda ---------------------+------------------------------------------------------ Reporter: knarf | Owner: Type: defect | Status: closed Priority: major | Milestone: Component: Other | Version: development version Resolution: fixed | Keywords: ---------------------+------------------------------------------------------
Comment (by sbrandt):
A related issue is that hwloc+Cactus doesn't compile on my platform without adding -lnuma to the final link.
trac@lists.einsteintoolkit.org
-
Einstein Toolkit