MacPort port upgrade fails with readline in /usr/local/lib in Snow Leopard 10.6.4

[vulcan_@…]

noticed that MacPorts needed an upgrade ( 1.9.1 installed), ran it and when it got to building {{{:info:build ===> making all in src/pextlib1.0 }}} I got this:

:info:build /usr/bin/gcc-4.2 -dynamiclib -pipe -O2 -arch x86_64 -arch i386 -W -Wall -pedantic      -Wl,-single_module Pextlib.o strsed.o fgetln.o md5cmd.o setmode.o xinstall.o fs-traverse.o strcasecmp.o vercomp.o filemap.o sha1cmd.o curl.o rmd160cmd.o sha256cmd.o readline.o uid.o tracelib.o tty.o readdir.o pipe.o flock.o system.o mktemp.o realpath.o -o Pextlib.dylib -L/System/Library/Frameworks/Tcl.framework/Versions/8.5 -ltclstub8.5 -arch x86_64 -arch i386   -lcurl  -lreadline
:info:build Undefined symbols:
:info:build   "_rl_username_completion_function", referenced from:
:info:build       _attempted_completion_function in readline.o
:info:build ld: symbol(s) not found
:info:build collect2: ld returned 1 exit status
:info:build Undefined symbols:
:info:build   "_rl_username_completion_function", referenced from:
:info:build       _rl_username_completion_function$non_lazy_ptr in readline.o
:info:build      (maybe you meant: _rl_username_completion_function$non_lazy_ptr)

google found reference to this exact set of error messages from a macports bug back in 2004? well .. long long ago. and the cause was a readline in /usr/local/lib

but i saw that the library it was trying to use was the libreadline from /usr/lib.
which is a link to libedit.dylib. OB some strange Mac OS X thing .. so i hid the link and tried again.

now it tried to use a libreadlib in /usr/local/lib and i got similar messages .. except that now the complaint was that there was no X86_86 architecture variant in the library. Which makes sense as my machine is a MacBook with and Intel Duo Core .. only 32 bit. The 64 bit variants should not be required.

i moved the local/lib files out of the way and tried again. Now it complained that it could not find the library at all. I check the Makefiles to see how dynamic libraries are being found. I have not set env var LD_LIBRARY_PATH and i do not have /etc/ld.so.conf and there is no sign that /opt/local/lib is being searched. Very Odd

i find the MacPorts Portfile and see that it is doing this

# Some of MacPort's default flags don't play well when we're building
# MacPorts itself: nuke them
configure.cppflags
configure.ldflags 

so it seems that when building the MacPorts itself the /opt/local/lib is not going to be searched. And there is no -L/opt/local/lib flag on the link command in the build log (attached). So the upgrade was still failing.

fumbling around in the Portfile and re-trying the upgrade command had the result that the whole thing was rebuilt from the configuration stage .. so the Makefiles all got rebuilt. Now when i re-tryed the upgrade it worked all the way through. Note the libreadline.dylib files in /usr/lib and /usr/local/lib are still all moved out of the way. I see that the command to build pexlib is now NOT using readline

/usr/bin/gcc-4.2 -dynamiclib -pipe -O2 -arch x86_64 -arch i386 -W -Wall -pedantic      -Wl,-single_module Pextlib.o strsed.o fgetln.o md5cmd.o setmode.o xinstall.o fs-traverse.o strcasecmp.o vercomp.o filemap.o sha1cmd.o curl.o rmd160cmd.o sha256cmd.o readline.o uid.o tracelib.o tty.o readdir.o pipe.o flock.o system.o mktemp.o realpath.o -o Pextlib.dylib -L/System/Library/Frameworks/Tcl.framework/Versions/8.5 -ltclstub8.5 -arch x86_64 -arch i386   -lcurl  

and this kind of sucks as the readline.c file is full of code that would love to be using it.

summary:

• MacPorts port is configured to require both 64 and 32 bit architectures in the libreadline.dylib on a Intel Duo Core running Snow Leopard that can only support 32 bit.
• (NB the port for readline is building libreadline for both architectures - that is bad and deserves its own bug report )
• MacPorts port leaves /usr/lib and /usr/local/lib in the search path for dynamic libs but can use only the MacPorts libreadline
• MacPorts port configuration only enables the use of readline when a libreadline.dylib is found in /usr/lib or /usr/local/lib .. which then causes pextlib to fail to build. /opt/local/lib is not searched for libraries and configure thus misses the only combination that will make use of the readline code in pextlib

[vulcan_@…]

log of first try at port upgrade MacPorts with /usr/lib/libreadlib in view

[jmroot (Joshua Root)]

Like all ports, it's not supported to have headers or libraries in /usr/local when building the MacPorts port. The port isn't a recommended way of getting MacPorts updates anyway, we just use it to build the disk images. You wouldn't have this problem with selfupdate.

[vulcan_@…]

I would like you to reconsider the decision on this defect. The reason is that we are not talking about files in /usr/local/ but rather about a version of the library installed in /usr/lib .. thus it is a system library that is part of Mac OS X .. so you can expect it to be there and MacPorts should be able to cope with it.

now i see a similar problem with #27316 .. /usr/lib/libcrypt is being taken before /usr/local and /opt/local is not even being looked at.

there is, according to me, a serious problem with MacPorts that could be fixed by adding -L/opt/local/lib to the link statements

i have been trying to learn enough about Portfiles to figure out how to do this when this problem shows up in a port .. and i would like to find out if there is a global setting that will fix this .. i have already tried

DYLD_LIBRARY_PATH=/opt/subversion/lib:/opt/local/lib:

in my shell env and that does not solve it.

so .. can we please have a good look at the situation.

[jmroot (Joshua Root)]

Base needs to be able to run before any ports are installed, so linking it against anything in /opt/local/lib is incorrect. Your log doesn't show any evidence of an architecture mismatch. My /usr/lib/libreadline.dylib doesn't contain a rl_username_completion_function symbol either, yet MacPorts builds fine. Removing the /usr/lib/libreadline.dylib link is a bad idea and done at your own risk. We can't do anything about headers from /usr/local/include being used because gcc is hardcoded to look there.

[cooljeanius (Eric Gallager)]

Cc Me!