#1006: ExternalLibraries' configure.sh scripts handle /usr inconistently ------------------------------------+--------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: Resolution: | Keywords: ExternalLibraries ------------------------------------+---------------------------------------

Comment (by eschnett):

The -L path should not contain those libraries that are searched by default. The reason is that otherwise these system locations are potentially searched first, making it impossible to replace libraries that already exist on the system. That is, if we want to provide a library via ExternalLibraries (e.g. HDF5) that already exists on the system, the linker may then choose the system library instead, which would be wrong.

/opt/local etc. (the "Mac OS X equivalents") are not part of the default search path, and should not be excluded.

/usr/include and /usr/local/include should be excluded from -I (include path).

#1006: ExternalLibraries' configure.sh scripts handle /usr inconistently ------------------------------------+--------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: Resolution: | Keywords: ExternalLibraries ------------------------------------+---------------------------------------

Comment (by hinder):

It turns out that this was not just a theoretical question; this is the exact cause of the problem I am fighting right now. The problem is not restricted to the system locations, it includes any location which might have more than one library present. Once any thorn has asked for that location to be added to the library path, you are at risk of getting all the libraries in that path from that location, irrespective of where you intended to get them from. For example, I have OpenMPI installed both by MacPorts in /opt/local and by the system in /usr. I also have HDF5 installed by MacPorts in /opt/local. Once HDF5 has added /opt/local/lib to the library path, there is no way to stop OpenMPI also being picked up from /opt/local/lib when I wanted the version in /usr.

Can one use an absolute path with "-l" to choose a specific version of a library?

#1006: ExternalLibraries' configure.sh scripts handle /usr inconistently ------------------------------------+--------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: Resolution: | Keywords: ExternalLibraries ------------------------------------+---------------------------------------

Comment (by hinder):

Should we think about implementing the symbolic link solution? I dread to think of how much work that will be, to get it correct on all systems for all libraries. But it should be rock-solid once it is done, and this will also provide a clear way to see exactly which libraries are used by a given configuration.

The problem seems to be that the "standard" method for handling libraries does not take account of having multiple versions of the library in different locations, some of which might contain more than one library.

#1006: ExternalLibraries' configure.sh scripts handle /usr inconistently ------------------------------------+--------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: Resolution: | Keywords: ExternalLibraries ------------------------------------+---------------------------------------

Comment (by rhaas):

Replying to [comment:1 eschnett]:

The -L path should not contain those libraries that are searched by

default. The reason is that otherwise these system locations are potentially searched first, making it impossible to replace libraries that already exist on the system. That is, if we want to provide a library via ExternalLibraries (e.g. HDF5) that already exists on the system, the linker may then choose the system library instead, which would be wrong.

/opt/local etc. (the "Mac OS X equivalents") are not part of the default

search path, and should not be excluded.

/usr/include and /usr/local/include should be excluded from -I (include

path).

This fails on OSX using homebrew where gcc does not by default look in /usr/local/XXX. So it does not find even the homebrew-installed hdf5 libraries unless one explicitly adds -L/usr/local/lib to the paths. The current gcc man page states: {{{ -Idir Add the directory dir to the head of the list of directories to be searched for header files. This can be used to override a system header file, substituting your own version, since these directories are searched before the system header file directories. However, you should not use this option to add directories that contain vendor-supplied system header files (use -isystem for that). If you use more than one -I option, the directories are scanned in left-to-right order; the standard system directories come after.

If a standard system include directory, or a directory specified with -isystem, is also specified with -I, the -I option is ignored. The directory is still searched but as a system directory at its normal position in the system include chain. This is to ensure that GCC's procedure to fix buggy system headers and the ordering for the "include_next" directive are not inadvertently changed. If you really need to change the search order for system directories, use the -nostdinc and/or -isystem options. }}} so removing the "standard" directories from -I is not required for gcc (but may be for other compilers). Not having to remove them ourselves would avoid putting information about the compiler behaviour into the configuration scripts for external libraries.

#1006: ExternalLibraries' configure.sh scripts handle /usr inconistently ------------------------------------+--------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: Resolution: | Keywords: ExternalLibraries ------------------------------------+---------------------------------------

Comment (by rhaas):

I was only thinking of -I (and -isystem) since there seems to be no similar logic for -L documented in gcc. My comment was not really clear about this, sorry. Since it fails for both "lib" and "include" dirs on homebrew, the comment evolved as I went along in making it work for me. So it seems that for "include" on can (using gcc) safely use -I but one cannot use -L for "lib" for the reasons Erik outlined.

LIBDIRS and SYS_INC_DIRS seem like better solutions though, so I have updated my changes for osx-yosemite-homebrew to instead use them. They will make homebrew look a little bit more like a Linux system.