atk build failed in macOS Sierra 10.12

[rickyzhang82 (Ricky Zhang)]

When upgrade outdated in macOS Sierra, atk build failed. It tried to link /opt/local/lib/libpcap.A.dylib for both 32bit and 64 bit. But run file command. It is 64bit only.

file /opt/local/lib/libpcap.A.dylib
/opt/local/lib/libpcap.A.dylib: Mach-O 64-bit dynamically linked shared library x86_64

[rickyzhang82 (Ricky Zhang)]

build failed log

[mf2k (Frank Schima)]

In the future, please use WikiFormatting.

This is not a clean build log. Please clean the port and try again. Then attach the new main.log file.

[rickyzhang82 (Ricky Zhang)]

Thanks for your correction.

I run clean all and then do update again.

Now I saw different port error dbus-python27 caused by the same reason. Please the log file. Should libpcap compile both 32 bit and 64 bit as well?

326 :info:build ld: in '/opt/local/lib/libpcap.A.dylib', file was built for x86_64 which is not the architecture being linked (i386): /opt/local/lib/libpcap.A.dylib for arch    itecture i386

[rickyzhang82 (Ricky Zhang)]

dbus-python27 log

[ryandesign (Ryan Carsten Schmidt)]

I saw this on my system too and didn't understand it. I don't see what libpcap has to do with atk or dbus-python27.

But then again, the problem didn't occur on the Sierra buildbot worker. So it might be intermittent.

Jeremy, do you have any ideas? This sounds like a problem from a year or two ago where the compiler (or linker?) would randomly try to add some unrelated library, which would often fail when building something universal, as is the case here, because the random library did not happen to be universal. I don't remember what the solution to that was at the time, do you?

[jeremyhu (Jeremy Huddleston Sequoia)]

:info:build libtool: link: /usr/bin/clang -Wl,-flat_namespace -Wl,-undefined -Wl,suppress -o .libs/_dbus_glib_bindings.so -bundle  .libs/module.o   -L/opt/local/lib -ldbus-glib-1 -ldbus-1 -lgobject-2.0 -lglib-2.0 -lintl  -Os -arch x86_64 -arch i386 -Wl,-headerpad_max_install_names -arch x86_64 -arch i386   -Wl,-exported_symbols_list,.libs/_dbus_glib_bindings-symbols.expsym
:info:build ld: in '/opt/local/lib/libpcap.A.dylib', file was built for x86_64 which is not the architecture being linked (i386): /opt/local/lib/libpcap.A.dylib for architecture i386
:info:build clang: error: linker command failed with exit code 1 (use -v to see invocation)

Change to that build directory and try running that link command with -v -Wl,-v arguments adde, and post the results.

[jeremyhu (Jeremy Huddleston Sequoia)]

And I don't recall the old issue, sorry.

[larryv (Lawrence Velázquez)]

Replying to ryandesign@…:

This sounds like a problem from a year or two ago where the compiler (or linker?) would randomly try to add some unrelated library, which would often fail when building something universal, as is the case here, because the random library did not happen to be universal. I don't remember what the solution to that was at the time, do you?

I think you’re talking about #36438, and the solution was that we gave up on it :-/

[larryv (Lawrence Velázquez)]

More specifically, we gave up on finding a general solution, and I hacked around it in the Perl ports by unsetting LIBRARY_PATH and removing the -L flags from the one compiler invocation that was triggering the link failure.

[ryandesign (Ryan Carsten Schmidt)]

Yes #36438 sounds like what I was thinking of.

The summary is that the linker is trying to use the MacPorts copy of libpcap, not because atk uses it, but because some system library atk uses uses it. (And this fails because we need a universal version, which the OS provides, but which the MacPorts copy on your system is not.) Jeremy thought this was a linker bug, and mentioned <rdar://problem/12450551>, but then later closed the ticket saying "There's not really anything we can do about this. This is how the toolchain works..." The buildbot did not encounter the problem because libpcap was not active at the time atk was built, so the linker successfully used the universal copy of the library provided by macOS. The build failed on our systems because we happened to already have a non-universal copy of libpcap installed by MacPorts. The workaround is to either deactivate libpcap for the duration of the atk build, or else to install libpcap universal. And if you see the same problem mentioning other ports such as dbus-python27, then the same applies to them: either install universal, or temporarily deactivate. Using MacPorts trace mode (sudo port -t install ...) may also work around it, but you may need to build MacPorts trunk from source, as the version of trace mode in MacPorts 2.3.x may not be sufficient for Sierra or El Capitan.

[ryandesign (Ryan Carsten Schmidt)]

2 other things to note:

1. Do you really need a universal version of atk? If not, install it without the universal variant to avoid the problem.
2. A universal binary of atk for Sierra is now available from our build server; if you sudo port clean atk and try again, you should receive the binary and not have to build it yourself.

[rickyzhang82 (Ricky Zhang)]

llvm-3.5-main.log

[rickyzhang82 (Ricky Zhang)]

Thanks for your detailed explanation. I don't know why I need atk at the very beginning. It must get installed because of some dependency.

I tried to follow jeremyhu's instruction. But after I run clean all/selfupdate/upgrade outdated. I hit another error in my port build.

Error: llvm-3.5 is not supported on macOS Sierra or newer.
Error: org.macports.fetch for port llvm-3.5 returned: unsupported platform
Please see the log file for port llvm-3.5 for details

I need to figure it out how to resolve it first. TBH, it is too early to claim that macport supports Sierra.

[ryandesign (Ryan Carsten Schmidt)]

MacPorts base works on Sierra; individual ports might not, and Jeremy has decided that llvm versions earlier than 3.7 do not work on Sierra.

Sierra was just released; you should expect it to take some weeks before maintainers have had a chance to upgrade to Sierra, figure out what problems exist with their ports, and figure out how to solve them.

I'm not sure what part of Jeremy's instructions led to an attempt to install llvm-3.5...

[jeremyhu (Jeremy Huddleston Sequoia)]

Why is atk (or a library that it's linking against) picking up libpcap in the first place? I don't see the usage.

As for llvm-3.5, uninstall it (and clang-3.5). Other dependents need to be updated to use newer versions of llvm.

[rickyzhang82 (Ricky Zhang)]

@jeremyhu

1. I'm scratching my head for this weird linking problem.
2. Next time please please please add some message to tell what need to be done if depreciated any packages. To any end users, printing the error without any further hints what need to do next is not quite helpful.

[rickyzhang82 (Ricky Zhang)]

After I removed llvm-3.5

sudo port uninstall --follow-dependents llvm-3.5

Finally, I can replicate the issue in another port - py27-gobject.

I checked libpcap. It disable universal explicitly in configure. For sure, I can remove libpcap temporarily. But this is a workaround.

[rickyzhang82 (Ricky Zhang)]

py27-gobject

[jeremyhu (Jeremy Huddleston Sequoia)]

Ryan jogged my memory.

Something that atk links against is linking against libpcap.dylib and the linker is resolving that to the MacPorts-provided one instead of the system-provided one.

Notice that /usr/lib/libnetwork.dylib links against libpcap.dylib, so maybe something is linking against libnetwork.dylib.

[ryandesign (Ryan Carsten Schmidt)]

Replying to zhangchaowang@…:

2. Next time please please please add some message to tell what need to be done if depreciated any packages. To any end users, printing the error without any further hints what need to do next is not quite helpful.

What would you have the llvm-3.5 port print? It doesn't know why you have asked to install it, only that you have asked to install it, and it knows that it is not compatible with your OS, so that's what it prints.

Replying to jeremyhu@…:

Something that atk links against is linking against libpcap.dylib and the linker is resolving that to the MacPorts-provided one instead of the system-provided one.

Notice that /usr/lib/libnetwork.dylib links against libpcap.dylib, so maybe something is linking against libnetwork.dylib.

I notice that /usr/lib/libnetwork.dylib links against /usr/lib/libpcap.A.dylib. Why isn't it considered a bug that the linker has decided to ignore that full path and go looking for it in the MacPorts directory instead? What's the point of using absolute paths for install_names if the linker ignores them?

[rickyzhang82 (Ricky Zhang)]

@ryandesign

My comment on llvm-3.5 is to improve usability. Because you are developers/maintainer, you know what need to do. However, for rest of users, it is essential to instruct them what need to do if shit happens. If you can embedded incompatible message, I'm sure you can add more message to instruct what need to be done. Regarding to what cause it, you can also figure it out if the port is already installed or not. Based on the available information, you know why you have asked to install it.

Macport is free software targeted for *nix users . But it doesn't mean we have to make it less user-friendly.

Just my 2 cents. No offense.

[jeremyhu (Jeremy Huddleston Sequoia)]

Replying to ryandesign@…:

I notice that /usr/lib/libnetwork.dylib links against /usr/lib/libpcap.A.dylib. Why isn't it considered a bug that the linker has decided to ignore that full path and go looking for it in the MacPorts directory instead? What's the point of using absolute paths for install_names if the linker ignores them?

It is a bug in the linker. That is <rdar://problem/12450551>, which I filed back in 2012.

Replying to zhangchaowang@…:

Based on the available information, you know why you have asked to install it.

The cause for the request to install or upgrade or any dependency information for that matter, is not available to the port during installation, so it's non-trivial to do what you are asking there.

[rickyzhang82 (Ricky Zhang)]

TBH, I'm an ignorant to the internal macport development matter. I have never written any port file from scratch.

I only fixed the patch and modified related in port files if things don't work.

But as an end user, I can run port installed command to list all installed port. If the port is already installed, running installation again implies it is an upgrade. Why is not trivial? Is it because port file doesn't provide such command/syntax to check if port is already installed?

[ryandesign (Ryan Carsten Schmidt)]

This ticket is about a problem building atk universal when some of its dependencies are not universal. As Jeremy said above, this is a linker bug; we cannot fix it in MacPorts.