perl5 builds incompletely with clang-3.4, resulting in dyld: lazy symbol binding failed: Symbol not found: _perl_tsa_mutex_lock

[snunez1 (Steve Nunez)]

This port, installed as a dependency for git, fails with:

:info:configure dyld: lazy symbol binding failed: Symbol not found: _perl_tsa_mutex_lock
:info:configure   Referenced from: /opt/local/lib/perl5/vendor_perl/5.26/darwin-thread-multi-2level/auto/Net/SSLeay/SSLeay.bundle
:info:configure   Expected in: flat namespace
:info:configure dyld: Symbol not found: _perl_tsa_mutex_lock
:info:configure   Referenced from: /opt/local/lib/perl5/vendor_perl/5.26/darwin-thread-multi-2level/auto/Net/SSLeay/SSLeay.bundle
:info:configure   Expected in: flat namespace
:info:configure sh: line 1: 72740 Trace/BPT trap          /opt/local/bin/perl5.26 Makefile.PL INSTALLDIRS=vendor CC="/opt/local/bin/clang-mp-3.9" LD="/opt/local/bin/clang-mp-3.9"

OS: 10.6.8 XCode 3.2 Upgraded to Clang 3.9

[snunez1 (Steve Nunez)]

log from build

[ryandesign (Ryan Carsten Schmidt)]

See #54364 where this was reported before. As was the case there, you are using LibcxxOnOlderSystems, which may be related (the build may be neglecting to pass the necessary -stdlib=libc++ flag on).

[SerpentChris (Chris Calderon)]

I've just run into this too, and I'm not sure how we are supposed to work around it. I tried building it again with the universal variant but it fails the same way. Do I need to uninstall perl entirely and rebuild with clang 3.9?

[ryandesign (Ryan Carsten Schmidt)]

Now that building with libc++ on older systems is the default as of MacPorts 2.6.0, we're seeing this type of failure with other perl ports on our buildbot, such as:

​https://build.macports.org/builders/ports-10.6_x86_64-builder/builds/4120/steps/install-port/logs/stdio

Ken, any ideas?

[kencu (Ken)]

The symbol is defined here <​https://perl5.git.perl.org/perl.git/blob/a975eebad:/perl.h#l3192> and implemented here <​https://perl5.git.perl.org/perl.git/blob/HEAD:/util.c#l6571>.

I see it is guarded by #ifdef PERL_TSA_ACTIVE so that might be relevant.

As to what that has to do with building on 10.6.8 with libc++, at this moment, I am not sure. I suppose the presence or absence of this symbol happens during the builld of perl and that's where we'd have to look for why it is or is not being included.

[kencu (Ken)]

Hmm. I seem to be having no problem installing this. So - ?

$ port -v installed | grep io-socket-ssl
  p5.26-io-socket-ssl @2.66.0_0 (active) platform='darwin 10' archs='noarch' date='2019-03-07T09:02:11-0800'
  p5.28-io-socket-ssl @2.66.0_0 (active) platform='darwin 10' archs='noarch' date='2019-10-04T18:33:48-0700'
  p5.30-io-socket-ssl @2.66.0_0 (active) platform='darwin 10' archs='noarch' date='2019-10-31T15:37:11-0700'

[kencu (Ken)]

and yet, for the one you linked to, I get the same error:

--->  Extracting DateTime-Locale-1.25.tar.gz
Executing:  cd "/opt/local/var/macports/build/_opt_macports-ports_perl_p5-datetime-locale/p5.30-datetime-locale/work" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/perl5/DateTime-Locale-1.25.tar.gz' | /usr/bin/gnutar --no-same-owner -xf - 
--->  Configuring p5.30-datetime-locale
Executing:  cd "/opt/local/var/macports/build/_opt_macports-ports_perl_p5-datetime-locale/p5.30-datetime-locale/work/DateTime-Locale-1.25" && /opt/local/bin/perl5.30 Makefile.PL INSTALLDIRS=vendor CC="/opt/local/bin/clang-mp-9.0" LD="/opt/local/bin/clang-mp-9.0" 
dyld: lazy symbol binding failed: Symbol not found: _perl_tsa_mutex_lock
  Referenced from: /opt/local/lib/perl5/vendor_perl/5.30/darwin-thread-multi-2level/auto/Variable/Magic/Magic.bundle
  Expected in: flat namespace

dyld: Symbol not found: _perl_tsa_mutex_lock
  Referenced from: /opt/local/lib/perl5/vendor_perl/5.30/darwin-thread-multi-2level/auto/Variable/Magic/Magic.bundle
  Expected in: flat namespace

sh: line 1: 99437 Trace/BPT trap          /opt/local/bin/perl5.30 Makefile.PL INSTALLDIRS=vendor CC="/opt/local/bin/clang-mp-9.0" LD="/opt/local/bin/clang-mp-9.0"
Command failed:  cd "/opt/local/var/macports/build/_opt_macports-ports_perl_p5-datetime-locale/p5.30-datetime-locale/work/DateTime-Locale-1.25" && /opt/local/bin/perl5.30 Makefile.PL INSTALLDIRS=vendor CC="/opt/local/bin/clang-mp-9.0" LD="/opt/local/bin/clang-mp-9.0" 
Exit code: 133

[kencu (Ken)]

If you configure the p5.30-variable-magic port, and then edit the Makefile to remove -undefined dynamic_lookup, and add in libperl.dylib instead then build it, you get:

LDDLFLAGS =  -mmacosx-version-min=10.6 -bundle /opt/local/lib/perl5/5.30/darwin-thread-multi-2level/CORE/libperl.dylib -L/opt/local/lib -Wl,-headerpad_max_install_names  -fstack-protector

/opt/local/bin/clang-mp-9.0  -mmacosx-version-min=10.6 -bundle /opt/local/lib/perl5/5.30/darwin-thread-multi-2level/CORE/libperl.dylib -L/opt/local/lib -Wl,-headerpad_max_install_names  -fstack-protector  Magic.o -arch x86_64 -Wl,-syslibroot,/ -o blib/arch/auto/Variable/Magic/Magic.bundle  \
	      \
	  
Undefined symbols for architecture x86_64:
  "_perl_tsa_mutex_lock", referenced from:
      _boot_Variable__Magic in Magic.o
      _XS_Variable__Magic_CLONE in Magic.o
      _XS_Variable__Magic__wizard in Magic.o
      _vmg_wizard_sv_free in Magic.o
      _vmg_wizard_sv_dup in Magic.o
      _xsh_teardown in Magic.o
      _xsh_teardown_late_simple_free in Magic.o
      ...
  "_perl_tsa_mutex_unlock", referenced from:
      _boot_Variable__Magic in Magic.o
      _XS_Variable__Magic_CLONE in Magic.o
      _XS_Variable__Magic__wizard in Magic.o
      _vmg_wizard_sv_free in Magic.o
      _vmg_wizard_sv_dup in Magic.o
      _xsh_teardown in Magic.o
      _xsh_teardown_late_simple_free in Magic.o
      ...
ld: symbol(s) not found for architecture x86_64

[kencu (Ken)]

Sorted out. In <​https://perl5.git.perl.org/perl5.git/blob/HEAD:/perl.h#l3038> the header checks for clang >= 3.6, and enables that macro only then. But on MacPorts, we're forcing perl to build with clang-3.4 for some reason on 10.6.8 (bootstrapping, no doubt), and so that test fails, and the #ifdef PERL_TSA_ACTIVE macro is disabled, ergo the symbol errors.

Building perl with a newer clang (I used clang-5.0) passes the tests, the symbol is enabled, and all is well:

$ port -v installed p5.30-datetime-locale
The following ports are currently installed:
  p5.30-datetime-locale @1.250.0_0 (active) platform='darwin 10' archs='x86_64' date='2019-10-31T17:02:29-0700'

[kencu (Ken)]

Josh and Mojca, adding you in.

[grumpybozo (Bill Cole)]

NOTE: This also affects p5-time-hires and almost certainly many other p5-* ports. The recent update to perl5.26 was built broken and packaged, causing breakage after installation of the new version.

A workaround for end-users is to force a local source build using a newer compiler, if one is already installed. e.g.:

      port -s upgrade perl5.26 configure.compiler=macports-clang-9.0

It seems to me that it would be best to remove the broken package(s) from the package server and provide some sort of public notice of the issue, since installing an updated Perl from MacPorts is hard to avoid and this issue gets reported by Perl as an error in add-on modules.

[kencu (Ken)]

With any luck, the changes I made recently to the clang builds should allow perl to build with clang-9.0 by default now -- at least that was the idea behind them. If that worked out as it was supposed to, we can just revbump perl and it should rebuild properly.

[grumpybozo (Bill Cole)]

This continues to be a problem with the perl5.28 binary package, which was built with clang 3.4 2020-06-03:

# perl5.28 -V
Summary of my perl5 (revision 5 version 28 subversion 3) configuration:
   
  Platform:
    osname=darwin
    osvers=10.8.0
    archname=darwin-thread-multi-2level
    uname='darwin snowleopard-i386.internal.macports.net 10.8.0 darwin kernel version 10.8.0: tue jun 7 16:33:36 pdt 2011; root:xnu-1504.15.3~1release_i386 i386 '
    config_args='-des -Dprefix=/opt/local -Dscriptdir=/opt/local/bin -Dvendorprefix=/opt/local -Dusemultiplicity=y -Dusethreads -Duseshrplib -Dcc=/opt/local/bin/clang-mp-3.4 -Dman1ext=1pm -Dman3ext=3pm -Dinstallstyle=lib/perl5 -Dman1dir=/opt/local/share/man/man1p -Dman3dir=/opt/local/share/man/man3p -Dsitebin=/opt/local/libexec/perl5.28/sitebin -Dsiteman1dir=/opt/local/share/perl5.28/siteman/man1 -Dsiteman3dir=/opt/local/share/perl5.28/siteman/man3 -Dvendorbin=/opt/local/libexec/perl5.28 -Dvendorman1dir=/opt/local/share/perl5.28/man/man1 -Dvendorman3dir=/opt/local/share/perl5.28/man/man3 -Dpager=/usr/bin/less -sR -Dperlpath=/opt/local/bin/perl5.28 -Dstartperl=#!/opt/local/bin/perl5.28 -Acppflags=-I/opt/local/include -Accflags=-pipe -Os  -Alddlflags=-L/opt/local/lib -Wl,-headerpad_max_install_names  -Aldflags=-L/opt/local/lib -Wl,-headerpad_max_install_names '
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=undef
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='/opt/local/bin/clang-mp-3.4'
 [...]

[ryandesign (Ryan Carsten Schmidt)]

I've inadvertently continued the discussion here: ​https://github.com/macports/macports-ports/pull/11534#issuecomment-881396393

[kencu (Ken)]

This is still a PITA. It was hard to work around today.

In the perl5 Porfile, we can certainly remove the part that adds clang-3.7 to extra dependencies. That might help some, but it was still a tricky thing to work around. I would like to get the clang-dependency PortGroup completely out of the perl5 Portfiles. I think all the clang portfiles have been fixed now to no longer need perl (perl5 is just for the analyzer that nobody uses anyway, at least on < 10.7).

I wanted per5.28 built +universal, for some reason, this kept happening:

$ sudo port -v -s install perl5.28 +universal configure.compiler=macports-clang-9.0
Error: Requested variants "" do not match those the build was started with: "+universal".
Error: Please use the same variants again, or run 'port clean perl5.28' first to remove the existing partially completed build.
Error: Problem while installing perl5.28
Error: Unable to execute port: upgrade clang-9.0 failed

I was unable to get past this Error: Requested variants "" do not match those the build was started with: "+universal". no matter what I tried.

Finally, I edited the clang-dependency PortGroup to make it basically be a no-op, then installed a non-universal version of perl5.28, and then force-upgraded it to +universal and that finally worked:

sudo port -v install perl5.28
sudo port -v -n -s upgrade --enforce-variants  perl5.28 +universal

With clang-dependency a no-op, perl5.28 built with clang-9.0, and is now fully functional.

[kencu (Ken)]

In 7665334d8d9670c9eb1304a35862d8ea4c5aa086/macports-ports (master):

clang-3.7: remove dependency on perl5 for < 10.7

this allows perl5 to build with clang-3.7 instead of clang-3.4
perl5 builds incompletely with clang versions < 3.7
closes: #57648

[kencu (Ken)]

In 57e876482001ea7b5407e1ee28d20cc1905b4cba/macports-ports (master):

perl5: Use newer compiler for perl_tsa_mutex_lock

Use a newer compiler (Apple clang 6.1 or later or open source clang 3.6
or later) to ensure that perl_tsa_mutex_lock gets compiled in, which is
needed if any module would ever get compiled with such a newer compiler.

See: Perl/perl5https://github.com/macports/macports-ports/pull/18991
See: ​https://github.com/macports/macports-ports/pull/11534

Closes: #57648