gawk: Library not loaded: /opt/local/lib/libreadline.7.dylib

[similate]

Running MacOS Mojave 10.14.2
XCode 10.1
Tried a selfupdate
Tried a port clean, then a port install
Then tried an unstall, a port clean, and a port install
Configure step is failing.
Failed to configure webp: configure failure: command execution failed

[similate]

Main log file with the error that occurred

[similate]

configure log -- mentioned at the end of the build but didn't see an error in it

[kencu (Ken)]

readline was just ​updated, and it affects so many ports there is always some breakage while everything gets revbumped:

info:configure dyld: Library not loaded: /opt/local/lib/libreadline.7.dylib
info:configure   Referenced from: /opt/local/bin/gawk

looks like you need to rebuild gawk

[similate]

Did a port install gawk; it installed and then the broken packages started installing including webp. Seems to be working; thank you!

[kencu (Ken)]

@ryandesign -- I'm not sure what to do with this ticket. I believe I recall the last time we updated readline you were not keen on revbumping gawk due to a gazillion rebuilds that would trigger.

[jmroot (Joshua Root)]

Gawk was already rev bumped for readline 8. The issue is that everything autoconf-based likes to use gawk (and ggrep and gsed) without being told to, so countless ports opportunistically use it (and so don't have a dependency on it which would cause it to be upgraded first) and break when it's broken, even though they build fine without it.

[ryandesign (Ryan Carsten Schmidt)]

There's nothing we can fix here.

[kencu (Ken)]

Oh, I recall what the issue was -- the fix is to give all the autoconf ports a dep on gawk, and that was unpalatable.

[ryandesign (Ryan Carsten Schmidt)]

Replying to kencu:

the fix is to give all the autoconf ports a dep on gawk,

And grep and gsed and conceivably perl and any other common utilities they might opportunistically use.

and that was unpalatable.

Quite.

[kencu (Ken)]

yeah. It always seems to be gawk that craps everyone out when readlne is bumped, tho. Havok for everyone until they work through it.

Is there no method by which this can be avoided? I wonder what the other 50 linux pkg mgrs do about this--must be just the same for them...

[ryandesign (Ryan Carsten Schmidt)]

Replying to kencu:

It always seems to be gawk that craps everyone out when readlne is bumped, tho.

Certainly. gawk depends on readline; gsed, grep and perl don't, but they have other dependencies, and they would experience the same problem if those other dependencies were bumped to a new library version.

Havok for everyone until they work through it.

Fortunately library version increases in those dependencies are rare. And many users who get binaries, and many users who build from source and happen to upgrade ports in the right order, won't experience any problems.

Is there no method by which this can be avoided? I wonder what the other 50 linux pkg mgrs do about this--must be just the same for them...

I don't know what Linux package managers do. But they are not in the same situation that MacPorts is in on macOS. macOS comes with an implementation of awk, grep, sed, and perl. We rely on that. Linux package managers manage all the software included with Linux; there is no non-package-managed copy of awk, grep, sed or perl that could be used, so I imagine they do declare all required dependencies.

If we are specifically concerned about the breakage when readline's major version changes about once a year (or similar breakage when ncurses' major version changes) we could change the gawk port not to use readline. I would expect gawk is not used in interactive mode very frequently, which is the only circumstance where readline support would be useful. Users who still want readline support could use rlwrap.

[Schamschula (Marius Schamschula)]

The problem seems to be with the rebuild order. As previously mentioned, gawk was revision bumped after the readline update.

However, if any other package is rebuilt that uses gawk anywhere in the process, before gawk itself is updated, we run into this problem.

When using postmaster in FreeBSD the rebuild/update is done recursively, insuring that build dependencies and dependencies are all up to date.

[ryandesign (Ryan Carsten Schmidt)]

Replying to Schamschula:

The problem seems to be with the rebuild order. As previously mentioned, gawk was revision bumped after the readline update.

However, if any other package is rebuilt that uses gawk anywhere in the process, before gawk itself is updated, we run into this problem.

Yes, that's correct.

When using postmaster in FreeBSD the rebuild/update is done recursively, insuring that build dependencies and dependencies are all up to date.

I am not familiar with FreeBSD ports, but the point is that in MacPorts, gawk is not a declared dependency of the ports where this problem is encountered. These ports use gawk opportunistically, as a consequence of the fact that they use an autotools build system.

[Schamschula (Marius Schamschula)]

Replying to ryandesign:

Replying to Schamschula:

The problem seems to be with the rebuild order. As previously mentioned, gawk was revision bumped after the readline update.

However, if any other package is rebuilt that uses gawk anywhere in the process, before gawk itself is updated, we run into this problem.

Yes, that's correct.

When using postmaster in FreeBSD the rebuild/update is done recursively, insuring that build dependencies and dependencies are all up to date.

I am not familiar with FreeBSD ports, but the point is that in MacPorts, gawk is not a declared dependency of the ports where this problem is encountered. These ports use gawk opportunistically, as a consequence of the fact that they use an autotools build system.

Indeed, most ./configure scripts will favor gawk over macOS awk, as the script will discover gawk first and go with it.

Right now FreeBSD is still using readline 7.0.5, but has already upgraded bash to 5.0.2. As I have gawk installed, I will observe how the update proceeds, when they do update readline.

It looks like they do something to deal with awk vs gawk. This morning I upgraded pkgconf and saw

checking for gawk... (cached) /usr/bin/awk

Nothing in the package's Makefile (yes, they use Makefiles instead of Portfiles...), or any global config file I could find.

[ryandesign (Ryan Carsten Schmidt)]

It is possible to ​set sitewide defaults for all configure scripts to use. MacPorts doesn't do that. I don't know if FreeBSD ports does.