libarchive port fails problem with -pthread argument to clang

[JustinCB]

The main log of the libarchive port

[JustinCB]

Cc Me!

[JustinCB]

Cc Me!

[JustinCB]

The problem seems to be that the libarchive port needs libpthread, but it doesn't provide the right argument to clang to get the library.

[JustinCB]

And that it thinks lzma is multithreaded, which prevents it from working.

[mf2k (Frank Schima)]

In the future, please Cc the port maintainers (port info --maintainers libarchive), if any. As reporter, you do not need to Cc yourself.

[tobypeterson]

Doesn't look like this has anything to do with pthreads. That's just a compilation warning.

Looks more like something sketchy going on with linking.

[tobypeterson]

CPATH='/Users/justin/include' LIBRARY_PATH='/Users/justin/lib'

That doesn't seem like a good idea. Try removing those and seeing if it works.

[JustinCB]

Apparently macports assumes that it's libraries are in the same directory as the system libraries. I installed macports in my home directory and added that to my path. It keeps it from accidentally overwriting system files and me from getting used to typing sudo all the time.

[neverpanic (Clemens Lang)]

Weird, the call of libtool seems to be correct:

./libtool  --tag=CC   --mode=link /usr/bin/clang ... -L/Users/justin/lib ... -L/Users/justin/lib ... -llzma ...

but the actual linker invocation generated by libtool isn't:

libtool: link: /usr/bin/clang ... -L/Users/justin/lib ... /usr/lib/liblzma.dylib ...

For some reason, libtool turns the valid spec into /usr/lib/liblzma.dylib, which doesn't match the headers and doesn't have multithread support, so linking fails. This might be a side effect of libtool filtering some uncommon arguments in an effort to try to be helpful ("surely you don't want to link against these files in your $HOME, let me helpfully remove those for you").

I assume you do have liblzma.dylib with those symbols in ~/lib?

[JustinCB]

~/liblzma.dylib exists. It is the version installed by macports with xz.

[JustinCB]

I compiled it without macports, and I moved *libiconv* in ~/lib/ to ~/lib/noexec and iconv.h in ~/include to ~/include/noinclude Is there a way to make macports used those that I compiled myself? To compile it, I also had to compile cmake.

[larryv (Lawrence Velázquez)]

Replying to JustinCB@…:

Is there a way to make macports used those that I compiled myself?

No.

To compile it, I also had to compile cmake.

Which part of this required CMake?

[JustinCB]

Configuring libarchive requires CMake(If it's from source). Macports CMake somehow requires libarchive. Macports libarchive has a bug that I filed this ticket for. That's why I'm trying to compile it myself right now.

[larryv (Lawrence Velázquez)]

Replying to JustinCB@…:

Configuring libarchive requires CMake(If it's from source). Macports CMake somehow requires libarchive.

It does not require CMake. Upstream provides an ​Autotools-based build system, which is what we use.

Macports libarchive has a bug that I filed this ticket for.

Debatable. I wouldn’t be surprised if your highly unorthodox configuration were causing the issue.

[JustinCB]

Replying to larryv@…:

Replying to JustinCB@…:

Configuring libarchive requires CMake(If it's from source). Macports CMake somehow requires libarchive.

It does not require CMake. Upstream provides an ​Autotools-based build system, which is what we use.

Macports libarchive has a bug that I filed this ticket for.

Debatable. I wouldn’t be surprised if your highly unorthodox configuration were causing the issue.

It isn't highly unorthodox, only slightly unorthodox. It is a part of the official macports source, but not the default. You can't use libraries you compiled yourself in macports, but you can use tools that you either compiled yourself or are included with the system(like CMake), and I was trying to build something that required CMake. It can use the CMake I compiled. I don't think this bug is in the libarchive build system, it is a bug in libtool that causes it to automatically link against the system libraries, even if you told it explicitly that you want it to link to different libraries. It might be so that it will make binaries that work on any system. It needs to do that to make binaries that work on any system because Mach-O links to a specific path instead of a path set by the environment.

[JustinCB]

I don't particularly like CMake, though. It's like a version of autotools that requires autotools to configure instead of generating a portable shell script.

[larryv (Lawrence Velázquez)]

Replying to JustinCB@…:

It isn't highly unorthodox, only slightly unorthodox.

Installing MacPorts somewhere under $HOME is not uncommon. Using $HOME directly as the prefix and overriding CPATH and LIBRARY_PATH is.

You can't use libraries you compiled yourself in macports, but you can use tools that you either compiled yourself or are included with the system(like CMake), and I was trying to build something that required CMake. It can use the CMake I compiled.

You can technically override MacPorts’ toolchain, yes, but we do not support doing so and are not obligated to accommodate such configurations.

I don't think this bug is in the libarchive build system, it is a bug in libtool that causes it to automatically link against the system libraries, even if you told it explicitly that you want it to link to different libraries.

There very well might be a bug in libtool, but it’s not quite what you are describing. Anyone using MacPorts in the standard fashion gets a libarchive that is not linked to the system libraries.

% otool -L /opt/local/lib/libarchive.13.dylib
/opt/local/lib/libarchive.13.dylib:
	/opt/local/lib/libarchive.13.dylib (compatibility version 16.0.0, current version 16.1.0)
	/opt/local/lib/libiconv.2.dylib (compatibility version 8.0.0, current version 8.1.0)
	/opt/local/lib/liblzo2.2.dylib (compatibility version 3.0.0, current version 3.0.0)
	/opt/local/lib/liblzma.5.dylib (compatibility version 8.0.0, current version 8.2.0)
	/opt/local/lib/libcharset.1.dylib (compatibility version 2.0.0, current version 2.0.0)
	/opt/local/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.6)
	/opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
	/opt/local/lib/libxml2.2.dylib (compatibility version 12.0.0, current version 12.4.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

[tobypeterson]

Yeah, something in libarchive's libtool must be doing something cute. The libtool invocation looks fine, it just somehow decides to use /usr/lib instead.

[JustinCB]

Installing MacPorts somewhere under $HOME is not uncommon. Using $HOME directly as the prefix and overriding CPATH and LIBRARY_PATH is.

You mean installing the port commands under $HOME is not uncommon, but $HOME as the prefix is? CPATH and LIBRARY_PATH are set to (prefix)/include and (prefix)/lib respectively.

You can't use libraries you compiled yourself in macports, but you can use tools that you either compiled yourself or are included with the system(like CMake), and I was trying to build something that required CMake. It can use the CMake I compiled.

You can technically override MacPorts’ toolchain, yes, but we do not support doing so and are not obligated to accommodate such configurations.

Macports defaults to using toolchains that already exist, even if they aren't Macports toolchains.

I don't think this bug is in the libarchive build system, it is a bug in libtool that causes it to automatically link against the system libraries, even if you told it explicitly that you want it to link to different libraries.

There very well might be a bug in libtool, but it’s not quite what you are describing. Anyone using MacPorts in the standard fashion gets a libarchive that is not linked to the system libraries.

% otool -L /opt/local/lib/libarchive.13.dylib
/opt/local/lib/libarchive.13.dylib:
	/opt/local/lib/libarchive.13.dylib (compatibility version 16.0.0, current version 16.1.0)
	/opt/local/lib/libiconv.2.dylib (compatibility version 8.0.0, current version 8.1.0)
	/opt/local/lib/liblzo2.2.dylib (compatibility version 3.0.0, current version 3.0.0)
	/opt/local/lib/liblzma.5.dylib (compatibility version 8.0.0, current version 8.2.0)
	/opt/local/lib/libcharset.1.dylib (compatibility version 2.0.0, current version 2.0.0)
	/opt/local/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.6)
	/opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
	/opt/local/lib/libxml2.2.dylib (compatibility version 12.0.0, current version 12.4.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

I think that the bug is that if it tries to link to a library that is not in a "standard" path, it will replace it with one that is in a "standard" path if one exists.

[larryv (Lawrence Velázquez)]

Replying to JustinCB@…:

You mean installing the port commands under $HOME is not uncommon, but $HOME as the prefix is?

Yes.

Macports defaults to using toolchains that already exist, even if they aren't Macports toolchains.

No, it does not. It uses tools in one of the following directories, expecting them to be provided by MacPorts or Apple.

• ${prefix}/bin
• ${prefix}/sbin
• /bin
• /sbin
• /usr/bin
• /usr/sbin

Modifying this search behavior or installing your own tools to those locations is absolutely unsupported.

I think that the bug is that if it tries to link to a library that is not in a "standard" path, it will replace it with one that is in a "standard" path if one exists.

What is “standard”? Who says that's a bug? It may very well be intentional.

[tobypeterson]

wacky config, marking wontfix