"dyld: Library not loaded: @rpath/libLLVM.dylib" in clang-3.8 and ld64-latest +llvm38

[RJVB (René Bertin)]

I have have just installed the binary llvm+clang 3.8 packages for OS X 10.9 and am running into issues with them.

With ld64_latest +llvm37 installed, I'm seeing ld-latest crashing and errors like this:

> clang++-mp-3.8 -O3 -march=native -v -o kk crc32test.cpp
clang version 3.8.0 (branches/release_38 262722)
Target: x86_64-apple-darwin13.4.0
Thread model: posix
InstalledDir: /opt/local/libexec/llvm-3.8/bin
 "/opt/local/libexec/llvm-3.8/bin/clang" -cc1 -triple x86_64-apple-macosx10.9.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -disable-free -disable-llvm-verifier -main-file-name crc32test.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu sandybridge -target-feature -sse4a -target-feature -avx512bw -target-feature +cx16 -target-feature -tbm -target-feature +xsave -target-feature -fma4 -target-feature -avx512vl -target-feature -prfchw -target-feature -bmi2 -target-feature -adx -target-feature -xsavec -target-feature -fsgsbase -target-feature +avx -target-feature -avx512cd -target-feature -avx512pf -target-feature -rtm -target-feature +popcnt -target-feature -fma -target-feature -bmi -target-feature +aes -target-feature -rdrnd -target-feature -xsaves -target-feature +sse4.1 -target-feature +sse4.2 -target-feature -avx2 -target-feature -avx512er -target-feature +sse -target-feature -lzcnt -target-feature +pclmul -target-feature -avx512f -target-feature -f16c -target-feature +ssse3 -target-feature +mmx -target-feature -pku -target-feature +cmov -target-feature -xop -target-feature -rdseed -target-feature -movbe -target-feature -hle -target-feature +xsaveopt -target-feature -sha -target-feature +sse2 -target-feature +sse3 -target-feature -avx512dq -target-linker-version 253.3 -v -dwarf-column-info -debugger-tuning=lldb -resource-dir /opt/local/libexec/llvm-3.8/bin/../lib/clang/3.8.0 -stdlib=libc++ -O3 -fdeprecated-macro -fdebug-compilation-dir /Users/bertin/work/src/new/Qt -ferror-limit 19 -fmessage-length 132 -stack-protector 1 -fblocks -fobjc-runtime=macosx-10.9.0 -fencode-extended-block-signature -fcxx-exceptions -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /var/folders/j1/1439ppj08xj8h6006s6drbq00000gs/T/crc32test-254929.o -x c++ crc32test.cpp
clang -cc1 version 3.8.0 based upon LLVM 3.8.0 default target x86_64-apple-darwin13.4.0
ignoring nonexistent directory "/usr/include/c++/v1"
#include "..." search starts here:
#include <...> search starts here:
 /opt/local/libexec/llvm-3.8/bin/../include/c++/v1
 /usr/local/include
 /opt/local/libexec/llvm-3.8/bin/../lib/clang/3.8.0/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
 "/opt/local/libexec/llvm-3.8/bin/ld" -demangle -dynamic -arch x86_64 -macosx_version_min 10.9.0 -o kk /var/folders/j1/1439ppj08xj8h6006s6drbq00000gs/T/crc32test-254929.o -lc++ -lSystem /opt/local/libexec/llvm-3.8/bin/../lib/clang/3.8.0/lib/darwin/libclang_rt.osx.a
dyld: Library not loaded: @rpath/libLLVM.dylib
  Referenced from: /opt/local/libexec/llvm-3.8/lib/libLTO.dylib
  Reason: image not found
clang: error: unable to execute command: Trace/BPT trap: 5
clang: error: linker command failed due to signal (use -v to see invocation)

When I build ld64-latest +llvm38 (non-default variant so by definition a local build) I'm seeing this:

%> otool -L /opt/local/bin/ld-latest 
/opt/local/bin/ld-latest:
        @rpath/libLTO.dylib (compatibility version 1.0.0, current version 3.8.0)
        /usr/lib/libxar.1.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)

%> /opt/local/bin/ld-latest
dyld: Library not loaded: @rpath/libLTO.dylib
  Referenced from: /opt/local/bin/ld-latest
  Reason: image not found
Trace/BPT trap
Exit 133

I think in both cases the error is that the referenced shared library isn't available in a directory that is on the standard rpath (or in ${prefix}/lib supposing that path is added to the rpath in all MacPorts builds). Indeed, when I symlink ${prefix}/libexec/llvm-3.8/lib/lib{LTO,LLVM}.dylib into /usr/local/lib the errors disappear.

[jeremyhu (Jeremy Huddleston Sequoia)]

ld64 needs to set its rpath. Working on this now.

[jeremyhu (Jeremy Huddleston Sequoia)]

r146522

[RJVB (René Bertin)]

Why not simply set the correct id in libLTO, libLLVM and libclang since they're not being installed in a standard location? I don't really see the point in forcing any user of those libraries (or simply libclang) to be obliged to figure out and add their location to the rpath ...

[jeremyhu (Jeremy Huddleston Sequoia)]

Because that is how upstream wants it.

[RJVB (René Bertin)]

In that case upstream should take the necessary steps to ensure that the appropriate rpath is added via llvm-config, the cmake scripts and whatever else.

[jeremyhu (Jeremy Huddleston Sequoia)]

Feel free to file a ticket at llvm.org.

[RJVB (René Bertin)]

Yeah, sure.

It'd just be a bit strange for me to argue that this should be done in (large) part for the sake of distribution/packaging systems like MacPorts; that argument probably stands to lose a lot of weight if not made by the maintainer of the clang port/package of such a distribution system.

Esp. since it's almost trivial to work around the whole issue using a few install_name_tool invocations in the post-destroot.

I'm taking this up with the main dev of KDevelop's Clang-based parser.

[jeremyhu (Jeremy Huddleston Sequoia)]

Yeah, we were setting the install name to the full path ourselves in older versions (eg: check out 0004-MacPorts-Only-Use-full-path-for-the-dylib-id-instead.patch in llvm-3.7), but in the transition to using the cmake build system, I decided to drop that and just go with matching upstream to limit our divergence. If you want to go down this route, I'd suggest advocating for a pkg-config file for libllvm and libclang rather than updating the llvm-config scripts.

[RJVB (René Bertin)]

Replying to jeremyhu@…:

r146522

Seems you missed something:

> otool -L /opt/local/libexec/ld64/ld-latest
/opt/local/libexec/ld64/ld-latest:
        @executable_path/../lib/libLTO.dylib (compatibility version 1.0.0, current version 3.8.0)
        /usr/lib/libxar.1.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)

I'd suggest advocating for a pkg-config file for libllvm and libclang rather than updating the llvm-config scripts.

Ideally dependent build systems would have to change as little as possible. But apparently this is not something new, which kind of begs the question why we're applications like ld64 didn't already have the proper addition to their rpath. With llvm designed to support multiple versions installed in parallel it shouldn't be that uncommon to have libraries with a version number in the name (libLTO, libclang) someplace *not* in the dynamic loader's standard search path, right?

[jeremyhu (Jeremy Huddleston Sequoia)]

Fixed the ones in libexec to have the expected @executable_path-relative linkage of libLTO in r146540.

As for why ld64 didn't have the -rpath already, that's because I forgot to add it to the Makefile because we used to not have libLTO be @rpath-relative.

I don't quite follow your train of thought in the rest of your comment. I suggest you followup on the mailing list because this isn't really the correct place to have a conversation about that.

[noloader (Jeffrey Walton)]

Please forgive my ignorance. What is the fix? What should I do to avoid the link problems I am experiencing?

Comment 8 (​https://trac.macports.org/ticket/50853#comment:8) seems to have the most promise, but it lacks details. That is, we are not told the exact command to execute to fix the problem.

My apologies for asking. I was referred to this bug from another one.

[jeremyhu (Jeremy Huddleston Sequoia)]

This was fixed months ago. You're looking for #51542.