clang sanitizer libraries have incorrect dylib id

[akimd (Akim Demaille)]

Hi,

It seems that clang expects the libraries for C++ to be at another place:

$ cat foo.cc                                                                                                                                           12
int main()
{
  return 1 << 34;
}

$ clang++-mp-3.9 -fsanitize=undefined foo.cc
foo.cc:3:12: warning: shift count >= width of type [-Wshift-count-overflow]
  return 1 << 34;
           ^  ~~
1 warning generated.
$ ./a.out
dyld: Library not loaded: /opt/local/libexec/llvm-3.9/lib/libclang_rt.ubsan_osx_dynamic.dylib
  Referenced from: /private/tmp/./a.out
  Reason: image not found
zsh: abort (core dumped)  ./a.out
$ ls  /opt/local/libexec/llvm-3.9/lib/libclang_*
zsh: no matches found: /opt/local/libexec/llvm-3.9/lib/libclang_*
$ ls /opt/local/libexec/llvm-3.9/lib/clang/3.9.1/lib/darwin
libclang_rt.10.4.a                      libclang_rt.eprintf.a                   libclang_rt.safestack_osx.a             libclang_rt.tsan_osx_dynamic.dylib
libclang_rt.asan_osx_dynamic.dylib      libclang_rt.osx.a                       libclang_rt.stats_client_osx.a          libclang_rt.ubsan_osx_dynamic.dylib
libclang_rt.cc_kext.a                   libclang_rt.profile_osx.a               libclang_rt.stats_osx_dynamic.dylib
$ DYLD_LIBRARY_PATH=/opt/local/libexec/llvm-3.9/lib/clang/3.9.1/lib/darwin ./a.out
foo.cc:3:12: runtime error: shift exponent 34 is too large for 32-bit type 'int'

[akimd (Akim Demaille)]

Note that SIP makes this even more troublesome: when running a program such as ./configure, DYLD_ variables are not preserved. Workaronds makes things more complex, and less portable.

[jeremyhu (Jeremy Huddleston Sequoia)]

Yeah, looks like the dylib ID is wrong:

~ $ otool -D /opt/local/libexec/llvm-3.9/lib/clang/3.9.1/lib/darwin/libclang_rt.asan_osx_dynamic.dylib 
/opt/local/libexec/llvm-3.9/lib/clang/3.9.1/lib/darwin/libclang_rt.asan_osx_dynamic.dylib:
/opt/local/libexec/llvm-3.9/lib/libclang_rt.asan_osx_dynamic.dylib

[jeremyhu (Jeremy Huddleston Sequoia)]

The executable's LC_RPATH is correct.

Load command 14
          cmd LC_RPATH
      cmdsize 32
         path @executable_path (offset 12)
Load command 15
          cmd LC_RPATH
      cmdsize 80
         path /opt/local/libexec/llvm-3.9/bin/../lib/clang/3.9.1/lib/darwin (offset 12)

We just need to figure out:

1. Why aren't the sanitizer dylibs getting installed with an @rpath-relative dylib id.
2. Optionally, figure out why the fixed path dylib id doesn't match the install path either.

[akimd (Akim Demaille)]

Ping? Not being able to use any of the sanitizers is disabling. Thanks!

[jeremyhu (Jeremy Huddleston Sequoia)]

You can use them just fine using DYLD_INSERT_LIBRARIES.

If you can answer either of those questions above and provide a solution, I'll integrate it. Otherwise, you're going to have to wait until someone else has time to look into the issue and provide such a solution. I currently don't have resources for that and won't for at least another month.

[hmijail (Horacio Mijail Antón Quiles)]

Just wanted to note that, as akimd mentioned already, the DYLD_* environment variables getting cleared by SIP make working around this issue quite difficult.

[akimd (Akim Demaille)]

Hi all,

FWIW, there is an easy workaround (that the package should implement imho): install a symlink. I've been using it successfully for quite a while now, with different versions of clang-*.

[hmijail (Horacio Mijail Antón Quiles)]

Issue still present in clang 5.0.

[jeremyhu (Jeremy Huddleston Sequoia)]

Yes, and I unfortunately don't see having time to get around to this issue any time soon, but if someone is able to provide a patch or can answer my points in comment #4, that would help.

[kencu (Ken)]

I think the logic is in this cmake file llvm-5.0.0.src/projects/compiler-rt/cmake/Modules/AddCompilerRT.cmake.

I note most of the compiler_rt modules are installed as STATIC, and only a few are SHARED.

Shared ones I found are:

clang_rt.asan
clang_rt.lsan
clang_rt.tsan
clang_rt.ubsan
clang_rt.stats

I would imagine they are all broken.

[kencu (Ken)]

If I'm understanding it correctly, the broken install names wind up here:

build/projects/compiler-rt/lib/asan/CMakeFiles/clang_rt.asan_osx_dynamic.dir/link.txt
build/projects/compiler-rt/lib/lsan/CMakeFiles/clang_rt.lsan_osx_dynamic.dir/link.txt
build/projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/link.txt
build/projects/compiler-rt/lib/ubsan/CMakeFiles/clang_rt.ubsan_osx_dynamic.dir/link.txt
build/projects/compiler-rt/lib/stats/CMakeFiles/clang_rt.stats_osx_dynamic.dir/link.txt

eg

/usr/bin/clang++ -pipe -Os -std=c++11 -stdlib=libc++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -Wall -std=c++11 -Wno-unused-parameter -DNDEBUG -arch x86_64 -arch x86_64h -dynamiclib -Wl,-headerpad_max_install_names  -stdlib=libc++ -lc++ -lc++abi -fapplication-extension -mmacosx-version-min=10.9 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -Wl,-U,___ubsan_default_options -Wl,-U,___sanitizer_free_hook -Wl,-U,___sanitizer_malloc_hook -Wl,-U,___sanitizer_report_error_summary -Wl,-U,___sanitizer_sandbox_on_notify -Wl,-U,___sanitizer_symbolize_code -Wl,-U,___sanitizer_symbolize_data -Wl,-U,___sanitizer_symbolize_demangle -Wl,-U,___sanitizer_symbolize_flush -L/opt/local/lib -Wl,-headerpad_max_install_names -o ../../../../lib/clang/5.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib -install_name /opt/local/libexec/llvm-5.0/lib/libclang_rt.tsan_osx_dynamic.dylib CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_clock.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_debugging.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_external.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_fd.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_flags.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_ignoreset.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interceptors.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface_ann.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface_atomic.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface_java.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_malloc_mac.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_md5.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_mman.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_mutex.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_mutexset.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_preinit.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_report.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_mutex.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_proc.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_report.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_thread.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_stack_trace.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_stat.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_suppressions.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_symbolize.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_sync.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interceptors_mac.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_libdispatch_mac.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_platform_mac.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_platform_posix.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_new_delete.cc.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_amd64.S.o CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_aarch64.S.o ../interception/CMakeFiles/RTInterception.osx.dir/interception_linux.cc.o ../interception/CMakeFiles/RTInterception.osx.dir/interception_mac.cc.o ../interception/CMakeFiles/RTInterception.osx.dir/interception_win.cc.o ../interception/CMakeFiles/RTInterception.osx.dir/interception_type_test.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_allocator.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_common.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_deadlock_detector1.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_deadlock_detector2.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_errno.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_flags.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_flag_parser.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_libc.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_libignore.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_linux.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_linux_s390.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_mac.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_persistent_allocator.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_platform_limits_linux.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_platform_limits_posix.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_posix.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_printf.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_common.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_freebsd.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_linux.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_mac.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stackdepot.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stacktrace.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stacktrace_printer.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stoptheworld_mac.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_suppressions.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer_libbacktrace.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer_mac.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer_win.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_tls_get_addr.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_thread_registry.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_win.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_termination.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_common_libcdep.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sancov_flags.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_coverage_libcdep_new.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_coverage_win_sections.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_linux_libcdep.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_mac_libcdep.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_posix_libcdep.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_stacktrace_libcdep.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_stoptheworld_linux_libcdep.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_symbolizer_libcdep.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_symbolizer_posix_libcdep.cc.o ../sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_unwind_linux_libcdep.cc.o ../ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_diag.cc.o ../ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_init.cc.o ../ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_flags.cc.o ../ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_handlers.cc.o ../ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_value.cc.o ../ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_handlers_cxx.cc.o ../ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_type_hash.cc.o ../ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_type_hash_itanium.cc.o ../ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_type_hash_win.cc.o -Wl,-rpath,/opt/local/libexec/llvm-5.0/lib 

with the install names set like this:

-install_name /opt/local/libexec/llvm-5.0/lib/libclang_rt.tsan_osx_dynamic.dylib

and the rpaths like this:

-Wl,-rpath,/opt/local/libexec/llvm-5.0/lib 

[kencu (Ken)]

One could hack in a reinplace fix for the portfile while we sort out the cmake error.

[kencu (Ken)]

Oh oh. I think we might be making this happen by setting the install name dir and the rpath in the portfile:

cmake.install_prefix ${sub_prefix}

# Adjust this once cmake-1.0.tcl is fixed:
#     https://github.com/macports/macports-ports/pull/103
# Also see:
#     https://llvm.org/bugs/show_bug.cgi?id=31425
configure.args-delete \
    -DCMAKE_INSTALL_NAME_DIR=${prefix}/lib \
    -DCMAKE_INSTALL_RPATH=${prefix}/lib \
    -DCMAKE_SYSTEM_PREFIX_PATH="${prefix}\;/usr"
configure.args-append \
    -DCMAKE_INSTALL_NAME_DIR=${cmake.install_prefix}/lib \
    -DCMAKE_INSTALL_RPATH=${cmake.install_prefix}/lib \
    -DCMAKE_SYSTEM_PREFIX_PATH="${cmake.install_prefix}\;${prefix}\;/usr"

setting the INSTALL_NAME_DIR and INSTALL_RPATH like this would apparently override anything done in llvm-5.0/clang-5.0 itself, so would result in the wonky names above.

It was done for a reason, though, for other dylibs that had the wrong names (see the llvm bug report). Have to see how to sort this out properly I guess, or just hack the fix in as above.

[kencu (Ken)]

Some useful references for this twice-a-year how-does-it-work-again issue:

<​http://log.zyxar.com/blog/2012/03/10/install-name-on-os-x/>

<​https://wincent.com/wiki/@executable_path,_@load_path_and_@rpath>

<​https://www.mikeash.com/pyblog/friday-qa-2009-11-06-linking-and-install-names.html>

[kencu (Ken)]

OK. Rebuilt clang-5.0, stripping out these two Portfile commands

    -DCMAKE_INSTALL_NAME_DIR=${cmake.install_prefix}/lib \
    -DCMAKE_INSTALL_RPATH=${cmake.install_prefix}/lib \

with that, clang-5.0 builds through to completion without any trouble. Running the example of the original poster, everything works as it should.

$ cat foo.cc
int main()
{
  return 1 << 34;
}


$ clang++-mp-5.0 -fsanitize=undefined foo.cc
foo.cc:3:12: warning: shift count >= width of type [-Wshift-count-overflow]
  return 1 << 34;
           ^  ~~
1 warning generated.

$ ./a.out
foo.cc:3:12: runtime error: shift exponent 34 is too large for 32-bit type 'int'

checking the executable shows the expected linking:

$ otool -L a.out
a.out:
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.0)
	@rpath/libclang_rt.ubsan_osx_dynamic.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)

and the rpath is set correctly

Load command 16
          cmd LC_RPATH
      cmdsize 72
         path /opt/local/libexec/llvm-5.0/lib/clang/5.0.0/lib/darwin (offset 12)

The sanitizer dylibs now have the proper names:

$ otool -D libclang_rt.asan_osx_dynamic.dylib
libclang_rt.asan_osx_dynamic.dylib:
@rpath/libclang_rt.asan_osx_dynamic.dylib

Now back to the original issue that led to this CMAKE_INSTALL_NAME in the first place:

$ otool -D libclang.dylib
libclang.dylib:
@rpath/libclang.dylib

$ otool -D libLLVM.dylib
libLLVM.dylib:
/opt/local/libexec/llvm-5.0/lib/libLLVM.dylib

$ otool -D libLTO.dylib
libLTO.dylib:
/opt/local/libexec/llvm-5.0/lib/libLTO.dylib

[kencu (Ken)]

everything seems to work properly and be fixed by taking those two -DCMAKE_INSTALL* lines out. I can push that through on clang-3.9 and up if that is the way Jeremy wants to go with this.

[jeremyhu (Jeremy Huddleston Sequoia)]

Can you send a github pull request? I'll give it a look there.

[kencu (Ken)]

Done. I didn't include clang-3.9 for now as I don't know how useful it is to fix it back there, and I didn't test what it might break in the older cmake rpath install logic of that version.

[kencu (Ken)]

In e75a4491bbaf8e8346068ac6c9f52e082e3634b3/macports-ports:

clang-4,5,devel: fix install names

clang installs some libraries in places other than
${cmake.install_prefix}/lib so we need to allow the
clang internal rpath mechanisms to set this

closes: #53299

[jeremyhu (Jeremy Huddleston Sequoia)]

Go ahead and make the same change for 3.9 as well. We'll be keeping it around for another year or so.

No need to do 3.8 since I'm getting ready to mark it obsolete.

[kencu (Ken)]

In fe10b2ff72776622c58214d00e243b131d75a469/macports-ports:

clang-3.9: fix install names

clang installs some libraries in places other than
${cmake.install_prefix}/lib so we need to allow the
clang internal rpath mechanisms to set this

closes: #53299

[kencu (Ken)]

This fix broke when <​https://github.com/macports/macports-ports/pull/103/commits/4cddb7db3a182f366d605e8db97944cd9d1e0e87> came through. Readjusting...

[ken-cunningham-webuse]

In b388e206e96c3a368da19664b4dd5582c769b5ac/macports-ports (master):

llvm/clang/lldb 3.9+ fix rpath handling, revbump

this was originally fixed in
<e75a4491bbaf8e8346068ac6c9f52e082e3634b3/macports-ports>

but once
<​https://github.com/macports/macports-ports/pull/103/commits/4cddb7db3a182f366d605e8db97944cd9d1e0e87>
was merged, the configure.args-delete in the portfile no longer deleted the install_name and install_rpath settings

this resulted in llvm/clang/lldb dylibs sometimes being installed with incorrect library names
and caused errors using the sanitizer libs and similar in clang

the PR restores the previous intended behaviour to allow the cmake scripts in llvm* to set the rpaths

see: #53299
closes: #57152
see: #58889