#1668: Cannot compile the ET on Datura ----------------------+----------------------------------------------------- Reporter: hinder | Owner: Type: defect | Status: new Priority: critical | Milestone: Component: Other | Version: development version Keywords: | ----------------------+----------------------------------------------------- Compilation of the ET fails on Datura while configuring MPI (see also #1667). When running the mpic++ command directly, I get {{{ [ianhin@login-damiana EinsteinToolkitGit]$ /cluster/openmpi/SL6/1.7.2/intel14/bin/mpic++ /cluster/openmpi/SL6/1.7.2/intel14/bin/mpic++: error while loading shared libraries: libimf.so: cannot open shared object file: No such file or directory }}} This happens because the compiler library is not on LD_LIBRARY_PATH: {{{ [ianhin@login-damiana EinsteinToolkitGit]$ ldd /cluster/openmpi/SL6/1.7.2/intel14/bin/mpic++ linux-vdso.so.1 => (0x00007fff2a1ff000) libopen-pal.so.5 => /cluster/openmpi/SL6/1.7.2/intel14/lib /libopen-pal.so.5 (0x00007fc755f21000) libm.so.6 => /lib64/libm.so.6 (0x00007fc755c90000) libdl.so.2 => /lib64/libdl.so.2 (0x00007fc755a8c000) librt.so.1 => /lib64/librt.so.1 (0x00007fc755884000) libnsl.so.1 => /lib64/libnsl.so.1 (0x00007fc75566a000) libutil.so.1 => /lib64/libutil.so.1 (0x00007fc755467000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fc755251000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fc755033000) libc.so.6 => /lib64/libc.so.6 (0x00007fc754ca0000) /lib64/ld-linux-x86-64.so.2 (0x00007fc75620e000) libimf.so => not found libsvml.so => not found libirng.so => not found libintlc.so.5 => not found }}} because when OpenMPI was compiled, its rpath was not set to point to the compiler installation directory.
This is because the compiler module is not loaded. The reason it is not loaded in the envsetup for the machine is that this this is specific to the version of the compiler, and one might want to use a different compiler via a different optionlist.
Since the optionlist is the only place for compiler-specific options, how can I set LD_LIBRARY_PATH so that mpicxx can run?
As a last resort, I suppose I can set all the required MPI variables in the optionlist, but MPI_DIR used to work with the old configure script. I would like it to work again now.
#1668: Cannot compile the ET on Datura -----------------------+---------------------------------------------------- Reporter: hinder | Owner: Type: defect | Status: new Priority: critical | Milestone: Component: Other | Version: development version Resolution: | Keywords: -----------------------+----------------------------------------------------
Comment (by rhaas):
Envsetup is also compiler specific. For example the Cray machines which use a common CC wrapper for all compilers decide which compiler is actually used based on the loaded modules. See eg. the bluewaters files in simfactory.
#1668: Cannot compile the ET on Datura -----------------------+---------------------------------------------------- Reporter: hinder | Owner: Type: defect | Status: new Priority: critical | Milestone: Component: Other | Version: development version Resolution: | Keywords: -----------------------+----------------------------------------------------
Comment (by hinder):
I think you are referring to the fact that since you need to load modules to use bluewaters, you are forced to have one machine definition per compiler, in order to use multiple envsetup variables? I think having a "machine" for each compiler (or other optional things, such as mpi version) is the right way to solve this. At the moment, this requires a lot of duplicated text in the machine definition files. This could be solved by allowing machine definitions to inherit from other machines. So bluewaters-cray.ini would just be {{{ [bluewaters-cray] inherit-from = bluewaters envsetup = source /etc/profile && module unload PrgEnv-cray PrgEnv- gnu PrgEnv-intel PrgEnv-pathscale PrgEnv-pgi && module load PrgEnv-cray && module load atp && module swap cce cce/8.1.5 && module load fftw/3.3.0.1 && module load hdf5/1.8.8 && module load papi/5.0.1 }}} Wanting to "mix and match" different options for a given machine and give them a name seems like quite a useful thing to be able to do in general. The name "machine" might be a little confusing if the concept is extended to mean a combination of machines and choices of the options to use on those machines.
#1668: Cannot compile the ET on Datura -----------------------+---------------------------------------------------- Reporter: hinder | Owner: Type: defect | Status: new Priority: critical | Milestone: Component: Other | Version: development version Resolution: | Keywords: -----------------------+----------------------------------------------------
Comment (by knarf):
So, the right way to solve this would be to load the right module on Datura, possibly providing different "machines" for different configurations? If so, this could be done, this ticket closed, and a request could be opened for the "include" feature in simfactory (maybe even with a patch? :) )
#1668: Cannot compile the ET on Datura -----------------------+---------------------------------------------------- Reporter: hinder | Owner: Type: defect | Status: closed Priority: critical | Milestone: Component: Other | Version: development version Resolution: fixed | Keywords: -----------------------+---------------------------------------------------- Changes (by hinder):
* status: new => closed * resolution: => fixed
Comment:
I have instead set the MPI options explicitly in the optionlist, so that you still don't have to load the module. It's all a bit of a mess; simfactory 3 will have something better.
trac@lists.einsteintoolkit.org
-
Einstein Toolkit