Use --x-includes and --x-libraries in certain ports to build against the 10.4u SDK X11 libraries

[david@…]

I have a MacPorts installation configured, in part, with --with-x11-prefix=/Developer/SDKs/MacOSX10.4u.sdk/usr/X11R6. I distribute binaries that are meant to run on both 10.4 and 10.5 so they must be linked against the 10.4 libraries. I had to modify the portfiles for cairo, gtk2, pango, and xrender to make them link properly.

[david@…]

Patch to use --x-includes and --x-libraries in certain ports

[raimue (Rainer Müller)]

CC'ing maintainers of cairo, pango and gtk2.

xrender does not have a maintainer.

[ryandesign (Ryan Carsten Schmidt)]

I don't think the X11 prefix should be set to /Developer/SDKs/MacOSX10.4u.sdk/usr/X11R6. Doesn't that then require that the MacOSX10.4u SDK be installed on the target system? That's not how the SDKs should be used. I think the correct solution is to set the X11 prefix to /usr/X11R6 and set the sysroot to /Developer/SDKs/MacOSX10.4u.sdk which should take care of linking X11 things with /Developer/SDKs/MacOSX10.4u.sdk/usr/X11R6 at build time. Have you tried setting universal_sysroot in macports.conf instead?

[david@…]

Replying to ryandesign@macports.org:

I don't think the X11 prefix should be set to /Developer/SDKs/MacOSX10.4u.sdk/usr/X11R6. Doesn't that then require that the MacOSX10.4u SDK be installed on the target system?

It doesn't seem to. I package the Nmap frontend ​Zenmap for Mac OS X using MacPorts to build its dependencies. The package runs universal on 10.4 and 10.5. I don't have a 10.4 machine to test on, so I relied on user reports to make things work on that platform. Before this change the application would crash on 10.4 with the following error:

ImportError: dlopen(/Applications/Zenmap.app/Contents/Resources/lib/python2.5/gtk/_gtk.so, 2): Library not loaded: /usr/X11/lib/libXau.6.dylib
Referenced from: /Applications/Zenmap.app/Contents/Resources/lib/python2.5/gtk/_gtk.so
Reason: image not found

Linking against the 10.4u X11 headers seems to avoid this dependency on libXau, and the resultant binary runs fine on 10.5, linking against the proper libraries in /usr/X11R6:

$ otool -L /Applications/Zenmap.app/Contents/Resources/lib/python2.5/gtk/_gtk.so /Applications/Zenmap.app/Contents/Resources/lib/python2.5/gtk/_gtk.so:
        @executable_path/../Frameworks/libgtk-x11-2.0.0.dylib (compatibility version 1201.0.0, current version 1201.9.0)
...
        @executable_path/../Frameworks/libpangocairo-1.0.0.dylib (compatibility version 2002.0.0, current version 2002.1.0)
        /usr/X11R6/lib/libSM.6.dylib (compatibility version 6.0.0, current version 6.0.0)
        /usr/X11R6/lib/libICE.6.dylib (compatibility version 6.3.0, current version 6.3.0)
        /usr/X11R6/lib/libX11.6.dylib (compatibility version 6.2.0, current version 6.2.0)
        @executable_path/../Frameworks/libcairo.2.dylib (compatibility version 20.0.0, current version 20.5.0)
...

(Edited for brevity.)

That's not how the SDKs should be used. I think the correct solution is to set the X11 prefix to /usr/X11R6 and set the sysroot to /Developer/SDKs/MacOSX10.4u.sdk which should take care of linking X11 things with /Developer/SDKs/MacOSX10.4u.sdk/usr/X11R6 at build time.

Forgive my ignorance. I'm new to Mac development. How should the SDKs be used?

Setting the sysroot works for most packages, but a few (like these) need special treatment. I have been maintaining a patch against the ports tree that allow building a 10.4-compatible universal environment. This is one of the changes in the patch.

Have you tried setting universal_sysroot in macports.conf instead?

Yes. The MacPorts installation I use to build releases is a Subversion checkout of 1.7.0, configured as follows:

export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11/bin
sudo mkdir /Library/Tcl/macports-universal-10.4
./configure --prefix=/opt/local-universal-10.4 --with-tclpackage=/Library/Tcl/macports-universal-10.4 \
  --with-universal-target=10.4 --with-universal-sysroot=/Developer/SDKs/MacOSX10.4u.sdk \
  --with-x11-prefix=/Developer/SDKs/MacOSX10.4u.sdk/usr/X11R6

Which sets universal_sysroot to

universal_sysroot       /Developer/SDKs/MacOSX10.4u.sdk

That takes care of most packages, but the ones mentioned were linking against the local 10.5 X11 libraries before this patch.

[afb@…]

Hmm, this wasn't a problem before when everything used /usr/X11R6 for x11prefix but you are right in that it will be when using different ones for Leopard/Tiger...

/Developer/SDKs/MacOSX10.4u.sdk should not be in the prefix, though. The reason it fails to link is because it couldn't find /usr/X11 in the Tiger installation.

[afb@…]

BTW you should be able to use /usr/X11R6 for both platforms, due to the compat symlink

[david@…]

Replying to afb@macports.org:

Hmm, this wasn't a problem before when everything used /usr/X11R6 for x11prefix but you are right in that it will be when using different ones for Leopard/Tiger...

/Developer/SDKs/MacOSX10.4u.sdk should not be in the prefix, though. The reason it fails to link is because it couldn't find /usr/X11 in the Tiger installation.

What about libXau? That's not on 10.4 anyway, right? It seems that linking against /usr/X11R6 on 10.5 makes the application incompatible with 10.4, while linking against the 10.4u SDK makes it compatible with both 10.4 and 10.5.

I tried building another MacPorts installation in /opt/local-x11, configured as above except with --with-x11-prefix=/usr/X11R6 (without this option the x11prefix will be /usr/X11).

Without the included patch, cairo, pango, and xrender link against the libraries in /usr/X11:

checking for X... libraries /usr/X11/lib, headers /usr/X11/include
...
 /usr/bin/gcc-4.0 -DHAVE_CONFIG_H -I. -I. -I. -I/opt/local-x11/include -isysroot
 /Developer/SDKs/MacOSX10.4u.sdk -I/opt/local-x11/include -I/usr/X11/include -DX
THREADS -O2 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch ppc -arch i386 -mmac
osx-version-min=10.4 -c Color.c  -fno-common -DPIC -o .libs/Color.o

And things link against libXau:

$ otool -L /opt/local-x11/lib/libXrender.dylib 
/opt/local-x11/lib/libXrender.dylib:
        /opt/local-x11/lib/libXrender.1.dylib (compatibility version 5.0.0, current version 5.0.0)
        /usr/X11/lib/libX11.6.dylib (compatibility version 9.0.0, current version 9.0.0)
        /usr/X11/lib/libXau.6.dylib (compatibility version 7.0.0, current version 7.0.0)
        /usr/X11/lib/libXdmcp.6.dylib (compatibility version 7.0.0, current version 7.0.0)
        /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.0.0)

After uninstalling everything and applying the patch, libraries are found in /usr/X11R6:

checking for X... libraries /usr/X11R6/lib, headers /usr/X11R6/include
...
 /usr/bin/gcc-4.0 -DHAVE_CONFIG_H -I. -I. -I. -I/opt/local-x11/include -isysroot
 /Developer/SDKs/MacOSX10.4u.sdk -I/opt/local-x11/include -I/usr/X11R6/include -
DXTHREADS -O2 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch ppc -arch i386 -mm
acosx-version-min=10.4 -c Color.c  -fno-common -DPIC -o .libs/Color.o

But things still link against libXau in /usr/X11:

$ otool -L /opt/local-x11/lib/libXrender.dylib 
/opt/local-x11/lib/libXrender.dylib:
        /opt/local-x11/lib/libXrender.1.dylib (compatibility version 5.0.0, current version 5.0.0)
        /usr/X11/lib/libX11.6.dylib (compatibility version 9.0.0, current version 9.0.0)
        /usr/X11/lib/libXau.6.dylib (compatibility version 7.0.0, current version 7.0.0)
        /usr/X11/lib/libXdmcp.6.dylib (compatibility version 7.0.0, current version 7.0.0)
        /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.0.0)

Additionally, I get this error, with or without the patch, so the build doesn't progress as far as gtk2:

ld: warning, duplicate dylib /opt/local-x11/lib/libz.1.dylib
ld: warning, duplicate dylib /opt/local-x11/lib/libiconv.2.dylib
ld: cycle in dylib re-exports with /usr/X11/lib/libGL.dylib
collect2: ld returned 1 exit status
ld: warning, duplicate dylib /opt/local-x11/lib/libz.1.dylib
ld: warning, duplicate dylib /opt/local-x11/lib/libiconv.2.dylib
ld: cycle in dylib re-exports with /usr/X11/lib/libGL.dylib
collect2: ld returned 1 exit status
lipo: can't open input file: /var/tmp//ccSu2Oew.out (No such file or directory)
make[4]: *** [libpangocairo-1.0.la] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Warning: the following items did not execute (for pango): org.macports.activate org.macports.build org.macports.destroot org.macports.install
Error: The following dependencies failed to build: pango shared-mime-info libxml2 p5-xml-parser perl5.8 tiff
Error: Status 1 encountered during processing.

So it seems that the universal_sysroot doesn't have an effect on where X11 headers and libraries are found? Sorry for the long reply.

[ryandesign (Ryan Carsten Schmidt)]

You should set universal_target to 10.4 and set universal_sysroot to /Developer/SDKs/MacOSX10.4u.sdk (because you want to target Mac OS X 10.4 and up) and set x11prefix to /usr/X11R6 (because that is where X11 is on Mac OS X 10.4). Well-behaved software should then look for X11 in ${universal_sysroot}{$x11prefix} at link time and in ${x11prefix} at runtime which should work for all users of Mac OS X 10.4 and later. I don't know that all ports you're interested in are thusly well-behaved because I haven't tried changing these variables from their default values. Actually since cairo doesn't currently build universal and pango depends on cairo and gtk2 depends on pango, xrender is the only port in this ticket that can currently be universal at all. (If you have information to the contrary please explain.)

I think it's probably a good idea to tell the ports where X11 is so I applied a slightly modified version of your patch in r38087.

[david@…]

Replying to ryandesign@macports.org:

You should set universal_target to 10.4 and set universal_sysroot to /Developer/SDKs/MacOSX10.4u.sdk (because you want to target Mac OS X 10.4 and up) and set x11prefix to /usr/X11R6 (because that is where X11 is on Mac OS X 10.4). Well-behaved software should then look for X11 in ${universal_sysroot}{$x11prefix} at link time and in ${x11prefix} at runtime which should work for all users of Mac OS X 10.4 and later.

That sounds like the way it should work. However I tried it (in a brand new ports tree in /opt/local-alt, with --with-applications-dir=/opt/local-alt/Applications and --with-frameworks-dir=/opt/local-alt/Library/Frameworks, but it didn't work. The libraries that are built are still linked against libXau. I'm going to keep using my current workaround (--with-x11-prefix=/Developer/SDKs/MacOSX10.4u.sdk/usr/X11R6).

Actually since cairo doesn't currently build universal and pango depends on cairo and gtk2 depends on pango, xrender is the only port in this ticket that can currently be universal at all. (If you have information to the contrary please explain.)

cairo builds universal for me using the custom patch against MacPorts that I use to build Zenmap. I submitted the cairo and openssl part of that patch in #15570. If you want to see the complete patch it's at ​svn://svn.insecure.org/zenmap/install_scripts/macosx/macports-1.7.0-universal.diff. Thanks to a lot of recent universal-compatibility work in MacPorts it's a lot smaller than it once was.

[(none)]

Milestone Port Enhancements deleted