poppler upgrade fails if older poppler is installed

[RobK88]

Unable to build poppler 0.77.0 on Mac OS X Lion

See attached log.

[RobK88]

P.S. I am configured to use libc++ on 10.7.5 as recommended.

Grinchs-Mac-Pro:~ grinch$ sudo port clean poppler
Password:
--->  Cleaning poppler
Grinchs-Mac-Pro:~ grinch$ sudo port upgrade poppler
--->  Computing dependencies for poppler
--->  Fetching distfiles for poppler
--->  Verifying checksums for poppler
--->  Extracting poppler
--->  Applying patches to poppler
--->  Configuring poppler
--->  Building poppler
Error: Failed to build poppler: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_poppler/poppler/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.
Grinchs-Mac-Pro:~ grinch$ 

[RobK88]

Please Note: The following is in my macport.conf file as recommended for Lion users in ticket:56811.

default_compilers macports-clang-5.0 macports-clang-4.0 macports-clang-3.9 macports-clang-3.8 macports-clang-3.7 clang

[ryandesign (Ryan Carsten Schmidt)]

What I notice is that the log says:

:info:build dyld: Symbol not found: _poppler_additional_action_type_get_type
:info:build   Referenced from: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_poppler/poppler/work/build/glib/tmp-introspectV_JG_y/Poppler-0.18
:info:build   Expected in: /opt/local/lib/libpoppler-glib.8.dylib

/opt/local/lib/libpoppler-glib.8.dylib is a file provided by the poppler port -- presumably, provided by whatever old version of poppler you currently have installed.

You can work around the problem by (forcibly) deactivating the currently installed version of poppler before trying to build the new one.

Dave/Michael: Please revert [18c09e26c90050596ad6c20e7c181de06c03e917/macports-ports]

[michaelld (Michael Dickens)]

@ryan: Sure will do. I'm traveling and at a conference this whole week, so please be patient!

[RobK88]

Ryan thanks for pin pointing the bug!

I uninstalled popller and then was able to successfully install the latest version of poppler 0.77.0.

I hope you fix the port soon for others who are trying to upgrade poppler to the latest version.

[michaelld (Michael Dickens)]

How about let's try to fix this issue, rather than just forcing the awkward conflict ...

Can we test out this "fix" in the poppler Portfile ... right before this code

configure.ldflags-append -liconv
gobject_introspection yes

add in the following:

# remove top-level paths, such that internal headers and libraries
# are used instead of any already-installed ones.
configure.ldflags-delete -L${prefix}/lib
configure.cppflags-delete -I${prefix}/include

In my reading of the output CMake build system, this should fix the issue. Can others test / verify?

[michaelld (Michael Dickens)]

​https://github.com/macports/macports-ports/pull/4575

[michaelld (Michael Dickens)]

In f4bbe69b73d95d6e9c76199337ca55cb81a9f6cf/macports-ports (master):

poppler: really fix self-build conflict

Closes: #58574

[ryandesign (Ryan Carsten Schmidt)]

I agree it's better to fix the conflict if we can figure out how. But that wasn't the fix. I still encountered the error while upgrading one of my machines.

[ryandesign (Ryan Carsten Schmidt)]

I hadn't upgraded that machine in awhile. It was upgrading from 0.69.0_0 to 0.77.0_0. So you should be able to reproduce the problem on your machine by first installing 0.69.0_0 and then trying to upgrade.

[michaelld (Michael Dickens)]

Well drat. Let me try again.

[michaelld (Michael Dickens)]

Has duplicate #58600

[michaelld (Michael Dickens)]

This is a sticky problem, which seems to be 2-pronged:

• The immediate build issue is caused somehow by the use of "${prefix}/bin/g-ir-scanner", which is provided by "port:gobject-introspection". It seems as though this binary when executed adds "-L${prefix}/lib" to link commands, which is what's causing the linking to the already-installed "poppler-glib" library. I'm trying to figure out how to coerce this binary into -not- adding that link command.

• "poppler"s CMake scripts need fixing to properly handle PKGCONFIG. Not a difficult change, but because it's a patch it's also susceptible to code changes down the road. It would be best upstreamed, which if I go this route I'll try to do & see what comes of it.

[kencu (Ken)]

has duplicate 58652

[ryandesign (Ryan Carsten Schmidt)]

Replying to michaelld:

• The immediate build issue is caused somehow by the use of "${prefix}/bin/g-ir-scanner", which is provided by "port:gobject-introspection". It seems as though this binary when executed adds "-L${prefix}/lib" to link commands, which is what's causing the linking to the already-installed "poppler-glib" library. I'm trying to figure out how to coerce this binary into -not- adding that link command.

That's not exactly the problem.

g-ir-scanner compiles and then runs a program called, in this case, Poppler-0.18.

The compiling finishes successfully. It specifies -L. and -lpoppler-glib which successfully finds the just-built libpoppler-glib.dylib in the current directory and links the program with it. It also specifies -L/opt/local/lib and -lgobject-2.0 -lgmodule-2.0 -lglib-2.0 -lintl to link the program with other MacPorts libraries.

The just-built libpoppler-glib.dylib has an install name of /opt/local/lib/libpoppler-glib.8.dylib but has not been installed to that location yet, so running Poppler-0.18 will load any old copy of the library that's already installed there, which will crash if that old library isn't sufficiently compatible (e.g. if 0.69 is installed but we were expecting 0.77).

If no version of the library has been installed yet, you would think that g-ir-scanner would error saying the library was not found, but that doesn't happen because it sets DYLD_FALLBACK_LIBRARY_PATH to the path to the just-built library.

It seems to me that the fix would be for g-ir-scanner to set DYLD_LIBRARY_PATH instead of DYLD_FALLBACK_LIBRARY_PATH, since we do want it to use the just-built library in all cases, even if one is already installed, not just as a fallback. In fact that's what g-ir-scanner used to do. It was switched to DYLD_FALLBACK_LIBRARY_PATH on your suggestion ​Marcus's suggestion because of a different problem.

That different problem should only occur if /opt/local/lib were also in DYLD_LIBRARY_PATH. So maybe the bug is that g-ir-scanner adds all -L paths to DYLD_LIBRARY_PATH. It should only add those paths to DYLD_LIBRARY_PATH that contain the just-built library(ies).

[michaelld (Michael Dickens)]

@ryandesign: that's fascinating! (really!) ... note that the FALLBACK was Marcus Calhoun-Lopez's suggestion, not mine. I didn't make it beyond what I've written here.

FYI: I'm heading off for vacation starting now for all practical purposes for 2 weeks. I was just going to revert the change to the poppler Portfile that started all of this .. but, if you think there's an alternative / better solution such as you describe, then I'm happy to hand this off to you to work on while I'm away.

[ryandesign (Ryan Carsten Schmidt)]

Replying to michaelld:

note that the FALLBACK was Marcus Calhoun-Lopez's suggestion, not mine.

Oops, then I'd better Cc him. Marcus, do you want to weigh in on this?

[koocsr]

I'm seeing this same issue, but for poppler-0.78.0, build fails on macos 10.13.6. Was doing "upgrade outdated" for the first time since it succeeded a couple weeks ago, on 7/1. Not sure if i should submit a new ticket, or if additional info from my "poppler/main.log" error will help.

[michaelld (Michael Dickens)]

Can you attach the logfile? Guessing it's the same issue.

Q: What version of poppler is currently installed that you're trying to upgrade from?

[koocsr]

poppler/main.log

[koocsr]

no version of poppler seems to be installed on my system, not in "/opt/local/bin/" ... it is being installed as part of some new dependency?

just attached the logfile.

[michaelld (Michael Dickens)]

what does "port installed poppler" return for you?

[michaelld (Michael Dickens)]

from the log file: this is the same issue.

The file "/opt/local/lib/libpoppler-glib.8.dylib" is installed, so "poppler" should also be installed and active.

[koocsr]

$ port installed poppler
The following ports are currently installed:
  poppler @0.52.0_0
  poppler @0.57.0_0 (active)

[michaelld (Michael Dickens)]

Interesting. When I updated from 0.57.0_0 to 0.58.0_0, it worked cleanly. Can you do the following:

sudo port clean poppler
sudo port selfupdate
sudo port upgrade outdated

and see if that works? Sometimes "cruft happens" and just needs to be cleaned out & the build tried again.

[koocsr]

just did that, but the same error persists:

"Symbol not found: _poppler_additional_action_type_get_type"

the new main.log file looks much the same as the one i uploaded before ... but i'll attach that one now, and terminal output

[michaelld (Michael Dickens)]

OK thanks for trying & for the info. Yes, same issue still.

Looks like you're running 10.13 ... let me try using that OS version & see if I can replicate the issue.

The easy solution for you is:

sudo port clean poppler
sudo port -f deactivate poppler
sudo port install poppler

should allow you to move forward.

[koocsr]

yes, that seems to have worked, thanks very much

[kencu (Ken)]

could we not use the deactivate hack instead to fix this?

[michaelld (Michael Dickens)]

Hmmm ... maybe use pre-configure instead of pre-activate? seems legit to me!

[ryandesign (Ryan Carsten Schmidt)]

Replying to kencu:

could we not use the deactivate hack instead to fix this?

We have not historically used the deactivate hack for this type of problem; instead, we use a self-conflict. The poppler port was using a self conflict until it was removed.

We should either reintroduce the self-conflict or better yet we should instead fix the problem as outlined in comment:15.

[michaelld (Michael Dickens)]

OK; I'm all for avoiding hacks and having ports build correctly while a prior version is installed.

Reviewing comment 15, I totally agree that using DYLD_LIBRARY_PATH is the way to go here. The FALLBACK won't work, since a library of the correct name is already installed. I use DYLD_LIBRARY_PATH regularly for this purpose (along with other such shell environment variables) to run a specific install of some port or something compiled from source but not yet installed (and/or when some other version is installed) -- this works like a charm!

As Marcus was the one who made the change, can we get his opinion on reverting it?

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #58788.

Michael, as I pointed out above, simply reverting the change would presumably reintroduce the problem that the change was intended to fix. I presume, as I stated above, that the problem is that all directories specified with -L flags are being added to DYLD_LIBRARY_PATH, which is not what we want. The correct thing to do would be to add only library paths that refer to directories inside the work directory to DYLD_LIBRARY_PATH. Of course I would certainly welcome input from Marcus as well.

[kencu (Ken)]

Seems to me that the issue here is really with poppler's versioning -- if they changed the library such that symbols have changed enough to break things, they should change the version sufficiently that there is no confusion about which library to link against. Then the one in ${prefix}/lib would be ignored.

[jmroot (Joshua Root)]

Adding a symbol doesn't warrant bumping the library major version, that's reserved for when you break backward compatibility by removing symbols or changing their semantics. Adding a symbol does mean the compatibility version should be increased, but that wouldn't avoid this problem, it would just change the error to "incompatible library version" rather than "symbol not found".

It really does just need to run with the same library it was built against.

[cjones051073 (Chris Jones)]

The issue of poppler not being able to build whilst an older version is installed has come back now that 0.80.0 is available.

Please, until a better solution is available please add the self-conflict to poppler, so at least users get a meaning message and not the current random build failure. Its particularly a pain as due to the license conflict with openssl poppler is not available as a binary tarball, so everyone has to build from source.

[michaelld (Michael Dickens)]

In 508a16373b11c97741666166b73bb13d269d3210/macports-ports (master):

poppler: restore build conflict with current installed poppler

Reverts: 18c09e26c90050596ad6c20e7c181de06c03e917/macports-ports

Closes: #58574

[jmroot (Joshua Root)]

This isn't really fixed; conflicts_build just documents that it's broken and prompts for a manual workaround. We should have a ticket open against gobject-introspection and one against poppler if its cmake scripts still need fixing as per comment:13.

[michaelld (Michael Dickens)]

Well from the perspective of the summary and description, it is fixed. Folks will have to work hard to have some already-installed poppler to have the issue found herein. :)

That said: Yes I do agree that the meta-issue is still at large & should probably have an issue open for it.

-That- said: I don't have time to deal with this, which is why I just pushed the build conflicts back into the port. I -always- prefer to be able to install a port regardless of what else is installed. It's more challenging sometimes to get the configuration and build scripts correct enough, but when done well it is maintainable and as future proof as can reasonably be expected.

[jmroot (Joshua Root)]

Sure, I understand that something needed to be done immediately. It's just unfortunate to have the record of the investigations into the underlying unresolved issue in a closed ticket.

[michaelld (Michael Dickens)]

Agreed! Well ... we can always reopen this ticket & change the title & embed the description into the discussion for legacy purposes ... that way we keep the discussion in the issue, and the issue title & keywords are relevant.

[koocsr]

I see that you closed out this issue as fixed, but FWIW, "upgrade updated" gave me the following:

--->  Computing dependencies for poppler
--->  Fetching archive for poppler
--->  Attempting to fetch poppler-0.80.0_0.darwin_17.x86_64.tbz2 from https://packages.macports.org/poppler
--->  Attempting to fetch poppler-0.80.0_0.darwin_17.x86_64.tbz2 from http://aus.us.packages.macports.org/macports/packages/poppler
--->  Attempting to fetch poppler-0.80.0_0.darwin_17.x86_64.tbz2 from http://ywg.ca.packages.macports.org/mirror/macports/packages/poppler
--->  Fetching distfiles for poppler
--->  Attempting to fetch poppler-0.80.0.tar.xz from https://distfiles.macports.org/poppler
--->  Verifying checksums for poppler
--->  Extracting poppler
--->  Applying patches to poppler
--->  Configuring poppler
Error: poppler cannot be built while another version of poppler is active.
Error: Please forcibly deactivate the existing copy of poppler, e.g. by running:
Error: 
Error:     sudo port -f deactivate poppler
Error: 
Error: Then try again.
Error: Failed to configure poppler: poppler is active
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_poppler/poppler/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.

which is fixed as before, with

sudo port clean poppler
sudo port selfupdate
sudo port upgrade outdated

but if poppler is required for some other dependency, after simply doing

sudo port -f deactivate poppler
sudo port upgrade outdated

it is not re-installed ...

[kencu (Ken)]

DON'T SHOOT ME, but -- this block appears to work to automatically uninstall an old poppler, if installed, during the build of the new poppler:

} else {
    # generation of Poppler-0.18.gir fails if previous version of poppler is active
    # appropriate for main poppler port only
    extract.asroot yes
    pre-extract {
        if {![catch {set installed [lindex [registry_active poppler] 0]}]} {
            registry_deactivate_composite poppler "" [list ports_nodepcheck 1]
        }
    }
}

You'll note I had to wrap it in the pre-extract block so I could run with extract.asroot yes, otherwise macports delivered an error trying to access the sqlite3 database with it's normal privs:

--->  Deactivating poppler @0.79.0_0
Error: Failed to fetch poppler: sqlite error: unable to open database file (14)
DEBUG: Error code: NONE
DEBUG: Backtrace: sqlite error: unable to open database file (14)
    while executing
"$post $targetname"
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_poppler/poppler/main.log for details.
DEBUG: sqlite error: unable to open database file (14)
    while executing
"registry::write {
                $port deactivate $imagefiles
                foreach file $files {
                    _deactivate_file $file
      ..."
    invoked from within
"try {
        # Block the TERM and INT signals to avoid being interrupted. Note that
        # they might already be block at this point because
     ..."
    (procedure "_deactivate_contents" line 39)
    invoked from within
"_deactivate_contents $requested [$requested files] $force"
    (procedure "deactivate" line 57)
    invoked from within
"deactivate $name "" "" 0 $optionslist"
    (procedure "portimage::deactivate_composite" line 3)

[kencu (Ken)]

FYI, the latest upgrade from poppler 0.81.0 to 0.82.0 proceeds fine, even if the older poppler is installed. Not sure if this is just a lucky upgrade, or if they fixed something in the cmake build finally...

[ryandesign (Ryan Carsten Schmidt)]

Replying to koocsr:

I see that you closed out this issue as fixed, but FWIW, "upgrade updated" gave me the following:

--->  Computing dependencies for poppler
--->  Fetching archive for poppler
--->  Attempting to fetch poppler-0.80.0_0.darwin_17.x86_64.tbz2 from https://packages.macports.org/poppler
--->  Attempting to fetch poppler-0.80.0_0.darwin_17.x86_64.tbz2 from http://aus.us.packages.macports.org/macports/packages/poppler
--->  Attempting to fetch poppler-0.80.0_0.darwin_17.x86_64.tbz2 from http://ywg.ca.packages.macports.org/mirror/macports/packages/poppler
--->  Fetching distfiles for poppler
--->  Attempting to fetch poppler-0.80.0.tar.xz from https://distfiles.macports.org/poppler
--->  Verifying checksums for poppler
--->  Extracting poppler
--->  Applying patches to poppler
--->  Configuring poppler
Error: poppler cannot be built while another version of poppler is active.
Error: Please forcibly deactivate the existing copy of poppler, e.g. by running:
Error: 
Error:     sudo port -f deactivate poppler
Error: 
Error: Then try again.
Error: Failed to configure poppler: poppler is active
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_poppler/poppler/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.

which is fixed as before, with

sudo port clean poppler
sudo port selfupdate
sudo port upgrade outdated

but if poppler is required for some other dependency, after simply doing

sudo port -f deactivate poppler
sudo port upgrade outdated

it is not re-installed ...

Yes, that is all now behaving as designed. I agree it is not great, but it is how all ports that have a self-conflict currently behave. Due to a defect in the poppler build system, or possibly in the way we are using it, poppler cannot build while certain other versions of itself are active, therefore you must deactivate poppler before upgrading it. We leave it to you to do that, rather than doing it automatically, because we are not privy to how you are using the ports you have installed with MacPorts and do not know if deactivating a port for the duration of an upgrade would cause problems in your use case. We don't know how long a port will take to build on your system, and we don't know how urgently you depend on this port or the ports that depend on it. Yes, a port is not considered outdated if it is inactive. You can still upgrade it explicitly by name, so to upgrade both the poppler you deactivated and all the other outdated ports you could run sudo port upgrade poppler outdated.

Ken, we have not used the deactivate hack for self-conflicts before. If you want to propose changing how we handle all self-conflicts, please open a discussion on the mailing list. Don't just change it for this one port. That decreases the consistency of the MacPorts experience.

The best thing we can do to improve this experience is fix the problem in the poppler build system so that the -I flags are in the right order and the self-conflict can then be removed.

[kencu (Ken)]

Poppler rebuilt this time for me with the previous version of poppler installed, without the deactivate hack. Maybe they fixed it? Or maybe just lucky this time?

Or maybe our default use of cmake's "-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON" is causing it. I haven't tried to fix it with the cmake commands.

(The deactivate hack "hack" does work here, as I jimmied it above, but the implementation is too messy to be acceptable, I agree).

[michaelld (Michael Dickens)]

Sometimes poppler can be updated while the prior version is in place; sometimes not. I doubt this update will work generically until the issue with comment:15 is dealt with.

[kencu (Ken)]

Just out of curiosity, have we tried this?

configure.args-replace -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=OFF

We force this to ON in MacPorts in the cmake PG, but there are times (like running tests) where I have to force it off to make things work properly. It's suppose to set the paths to the in-tree libs until the software is actually installed.

<​http://docs1.w3cub.com/cmake~3.5/variable/cmake_build_with_install_rpath/>

[kencu (Ken)]

According to <​https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/RPATH-handling> the linker will search for dependencies in the following order:

1. DYLD_LIBRARY_PATH - an environment variable which holds a list of directories
2. RPATH - a list of directories which is linked into the executable. These can contain @loader_path and @executable_path.
3. builtin directories - /lib /usr/lib
4. DYLD_FALLBACK_LIBRARY_PATH - an environment variable which holds a list of directories

[kencu (Ken)]

Replying to kencu:

Just out of curiosity, have we tried this?

configure.args-replace -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=OFF

Doesn't fix the linking.

[ryandesign (Ryan Carsten Schmidt)]

Ken, there is no need for any further investigations. I've explained the problem and the solution in comment:15. If you want to resolve this ticket, comment:15 will tell you what needs to be done.

[kencu (Ken)]

hmm, yes. This damned Poppler issue is a huge pain in the rear -- it seems the authour of poppler likes to update it every two weeks, and it never updates without manual intervention. Ugh.

I am liking my "force the old one out" approach more and more as NOBODY seems to be able to figure out how to make it build against the newly built rather than previously installed version, despite all the DYLD_LIBRARY_PATH shenanigans that just don't fix the (years old) issue....

[kencu (Ken)]

first suggestion is that we don't bother updating poppler any more often than twice a year, whatever crazy schedule the authour wants to to set.

[ryandesign (Ryan Carsten Schmidt)]

Do you have any evidence that anyone has tried to do what I suggested in comment:15?

[kencu (Ken)]

No forensic evidence is available here for such an event or lack thereof. Just lots of time gone by, no progress made.

[ryandesign (Ryan Carsten Schmidt)]

The first step should be to report the problem to the developers. I've now done that: ​https://gitlab.gnome.org/GNOME/gobject-introspection/-/issues/335

[mascguy (Christopher Nielsen)]

I was just bitten by this for the first time. And after the initial "huh?" reaction, it was easy enough to forcibly deactivate the port first, and then upgrade it.

That being said, until the Gnome folks fix the issue reported by Ryan, I'd vote for Ken's "force the old one out" approach. MacPorts is an amazing project/resource, and is used by countless users... both technical folks like all of us, as well as non-techies.

And in the interest of providing the best MacOS user experience possible, wouldn't we want things to "just work?"

[ryandesign (Ryan Carsten Schmidt)]

We do want things to "just work". That's the reason why we don't deactivate ports at unexpected times. If we were to deactivate poppler before building it, and poppler were to then fail to build for some reason, poppler would remain deactivated, causing any software the user had installed that required poppler to fail to work. That would in turn probably generate more tickets for us to deal with. We would rather have the user take responsibility for deactivating the port so that they have some awareness of the possible need to reactivate it manually later.

[mascguy (Christopher Nielsen)]

Makes sense Ryan. Let's hope the Gnome folks fix this sooner, rather than later...

[mascguy (Christopher Nielsen)]

So thinking more about this, would it be feasible to forcibly deactivate (rather than remove) Poppler during the upgrade... and then simply reactivate the old version if the build fails?

[ryandesign (Ryan Carsten Schmidt)]

Replying to mascguy:

and then simply reactivate the old version if the build fails?

How would that be accomplished?

Instead, we should focus efforts on fixing the real problem: the bug in gobject-introspection.

[mascguy (Christopher Nielsen)]

Replying to ryandesign:

How would that be accomplished?

Ken provided a proposed resolution via the port file, in comment:45.

As a software developer, I can appreciate not wanting to implement any hacks. That said, Ken's solution looks clean. Enough so, that I wouldn't consider it a hack... but rather, behavior that's necessary due to the state of the port.

Would you be willing to reconsider Ken's solution? Please? :-)

[mascguy (Christopher Nielsen)]

Also, just to help you understand why I'm pushing for this...

I maintain four MacOS versions: 10.12, 10.13, 10.14, and 10.15. That's necessary for development reasons, as well as to deal with legacy software that won't run on the latest MacOS releases. Once 10.16 is released and stable, I'll have five.

Anyway... Across those four versions, I maintain a matching set of ports. Each of which needs to be updated at the same time, to ensure consistency. And over the past week, there have been two new versions of the Poppler port released. And in both instances, the upgrades required futzing with manual deactivation... four times. Not fun!

Bear in mind that I'm not complaining about the frequent updates. Indeed, the great thing about MacPorts, is that you folks keep everything up-to-date relative to the various projects. And that's awesome!

However, this particular case is irritating. Enough so, that I'm prepared to write a quick wrapper script to patch the Poppler port file with Ken's change, as part of my MacPorts upgrade workflow.

But as a user, it's frustrating that I'm now at that point.

Given that the proposed change isn't fragile -- it doesn't involve a moving target, such as patching the makefile for 'gobject-introspection' -- I simply can't understand the resistance...

[ryandesign (Ryan Carsten Schmidt)]

Replying to mascguy:

Replying to ryandesign:

How would that be accomplished?

Ken provided a proposed resolution via the port file, in comment:45.

He provided a proposal that deactivates poppler before the build if it's active. I agree that's possible. He did not provide a way to reactivate the old port if the current port should fail to build, which isn't so easy, since the moment the build fails, the port no longer has control, control is handed over to base, so the port does not get an opportunity to reactivate the old port.

I would rather we focus on fixing the gobject-introspection bug. Unfortunately nobody has responded to any of the questions I asked the gobject-introspection developers. Somebody with time may need to immerse themselves in the gobject-introspection source code to understand enough of how it works to be able to answer the questions and then fix the bug.

poppler can't be the only software affected by this bug. Fixing the bug will likely solve problems in other software that we don't even know yet are associated with this bug.

[mascguy (Christopher Nielsen)]

Replying to ryandesign:

He provided a proposal that deactivates poppler before the build if it's active. I agree that's possible. He did not provide a way to reactivate the old port if the current port should fail to build, which isn't so easy, since the moment the build fails, the port no longer has control, control is handed over to base, so the port does not get an opportunity to reactivate the old port.

Ah, fair enough.

Okay, so here's the million-dollar question: Would there be interest in enhancing the MacPorts base, to allow more customized error handling via the port file? (Assuming the enhancements were contributed by someone... like myself.)

The idea being that, while it would certainly be useful in this particular case, it could also be useful for other things down the road.

I realize more discussion would be warranted via the mailing list, including a more formal proposal. But just wanted to ask that question before going further...

[kencu (Ken)]

The gobject-introspection source code is quite readable and hackable.

I think, as Ryan said, we just need to use DYLD_LIBRARY_PATH instead of DYLD_FALLBACK_LIBRARY_PATH, and have the code stop adding -L/opt/local/lib to that path, which seems both unnecessary and troublesome.

[kencu (Ken)]

It would be really nice to have a much simpler piece of software to work with to fix this bug rather than poppler, which takes forever to build and does not reliably show the bug.

[ryandesign (Ryan Carsten Schmidt)]

Chris, I don't know what you're referring to when you ask about more customized error handling. If you're suggesting MacPorts base should give portfiles an opportunity to run code after a build fails, I don't know what that would be good for, except to implement the workaround you've discussed in this ticket which I do not recommend implementing. Since a portfile is a Tcl file it could technically already implement anything you want. poppler could override the build phase so that failures are ignored, and then implement its own detection of the failure condition and respond appropriately. I do not want to spend any time overcomplicating the poppler port to implement this; we should fix the real problem instead.

Ken, in the upstream bug report I pointed out that libdazzle is such a much simpler piece of software. If you don't like that one, you can check any other ports that depend on gobject-introspection. As far as "reliably show[ing] the bug", one only needs to add some printf statements to g-ir-scanner to see what paths it was invoked with or what it will set the DYLD variable to to see whether it is right or wrong.

[mascguy (Christopher Nielsen)]

Replying to ryandesign:

Since a portfile is a Tcl file it could technically already implement anything you want. poppler could override the build phase so that failures are ignored, and then implement its own detection of the failure condition and respond appropriately.

Ryan, thanks for the clarification.

I added a comment to the issue you logged against the 'gobject-introspection' project, asking the developers to look at the issue. Perhaps if enough of us speak up, they might finally respond...

[michaelld (Michael Dickens)]

The issues are quite multi-faceted, and thus require quite a bit of testing / verification! The critical PR is in now: ​https://github.com/macports/macports-ports/pull/8779 .. I'll get in the other one to Poppler itself shortly, but I don't think this one is critical to the in-build execution.

It's easy to replicate -- just install a much older poppler version from current; I'm using 0.79.0_0. Then with that active, try to build current poppler. With an old enough poppler, the new-poppler build will fail because of the issues discussed herein. Tracking down the sources of the errors isn't trivial, but, as @kencu notes:

Replying to kencu:

The gobject-introspection source code is quite readable and hackable.

Agreed! The scanner is written in Python, which is nice and easy to work with since one can make changes without having to recompile / reinstall; makes debugging -so- much easier! Still not trivial, but much better than C/C++!

I think, as Ryan said, we just need to use DYLD_LIBRARY_PATH instead of DYLD_FALLBACK_LIBRARY_PATH ...

You are correct ... and this is an almost trivial change.

... and have the code stop adding -L/opt/local/lib to that path, which seems both unnecessary and troublesome.

This is actually OK, it turns out. The GO-I scanner adds all of the build / source library directories -before- any system ones, and CC takes these in the order provided, so this entry works out OK -- at least in my current testing. Removing this entry is non-trivial, but doable. I'm going to add this to my fixes, since it seems like the right thing to do.

There is 1 more step for completion is to reorder the -I entries so that those in the build / source are found before those in the system. poppler's CMake code does this correctly for its own C/C++ compiling, but then totally borks it in the interface to gobject-introspection. Doing this isn't too difficult; it's tracking down where to do it in the first place that's the pain!

Note that since we're using an older version of GO-I (pre-meson @ 1.60.2 ; current is 1.66.1), so any fixes here may or not be relevant to current GO-I. If/when we decide to upgrade GO-I, it will involve retesting and possible fixing up the patches we come up with to fix this issue for us ... which makes upgrading that much more challenging!

[kencu (Ken)]

To be honest, nobody upstream likely uses Macs, so they are looking for users to fix it.

Be sure that whatever we propose also fixes the issue that started all this in the first place (not from a MacPorts user):

<​https://mail.gnome.org/archives/gtk-osx-users-list/2018-February/msg00000.html>

[kencu (Ken)]

Replying to michaelld:

... and have the code stop adding -L/opt/local/lib to that path, which seems both unnecessary and troublesome.

This is actually OK, it turns out.

Marcus found out that setting this while using DYLD_LIBRARY_PATH overrides system frameworks by making them use newer macports libraries like libjpeg, and so all the links fail. That's why he suggested using FALLBACK, which has it's own problems.

[michaelld (Michael Dickens)]

Replying to kencu:

To be honest, nobody upstream likely uses Macs, so they are looking for users to fix it.

Be sure that whatever we propose also fixes the issue that started all this in the first place (not from a MacPorts user):

<​https://mail.gnome.org/archives/gtk-osx-users-list/2018-February/msg00000.html>

Yes my tweaks address that issue. This is exactly what I was experiencing: the in-build executable failed because the DLYD found the library in MP_PREFIX/lib before the one in-build, and that library was too old to provide the necessary symbols. The DYLD_FALLBACK_LIBRARY_PATH environment variable doesn't work here, since the library is found in the executable (confirmed by otool -L); one has to use the full DYLD_LIBRARY_PATH directly.

[michaelld (Michael Dickens)]

Replying to kencu:

Replying to michaelld:

... and have the code stop adding -L/opt/local/lib to that path, which seems both unnecessary and troublesome.

This is actually OK, it turns out.

Marcus found out that setting this while using DYLD_LIBRARY_PATH overrides system frameworks by making them use newer macports libraries like libjpeg, and so all the links fail. That's why he suggested using FALLBACK, which has it's own problems.

Correct all around! So what I do is use DYLD_LIBRARY_PATH for the in-build paths and DYLD_FALLBACK_LIBRARY_PATH (and LIBRARY_PATH itself, which honestly is probably more important) for non-in-build-paths -- for example MP paths. I have the PKGCONFIG command return 2 variables instead of 1: the non"-L" entries and the "-L" entries. These are then handled separately, which means we've removed all of the system "-L" entries from the link command and instead rely on DYLD_FALLBACK_LIBRARY_PATH and LIBRARY_PATH. This in my testing was the most reliable and cleanest way to get these builds to work as well as be executable in-build.

[mascguy (Christopher Nielsen)]

Replying to michaelld:

Correct all around! So what I do is use DYLD_LIBRARY_PATH for the in-build paths and DYLD_FALLBACK_LIBRARY_PATH (and LIBRARY_PATH itself, which honestly is probably more important) for non-in-build-paths -- for example MP paths. I have the PKGCONFIG command return 2 variables instead of 1: the non"-L" entries and the "-L" entries. These are then handled separately, which means we've removed all of the system "-L" entries from the link command and instead rely on DYLD_FALLBACK_LIBRARY_PATH and LIBRARY_PATH. This in my testing was the most reliable and cleanest way to get these builds to work as well as be executable in-build.

Michael, is this straightforward enough, that you can provide a patch to fix this?

[michaelld (Michael Dickens)]

FYI the PR for this issue is ​https://github.com/macports/macports-ports/pull/8780 ... I'm not sure I noted it here yet ...

[michaelld (Michael Dickens)]

Replying to mascguy:

Michael, is this straightforward enough, that you can provide a patch to fix this?

Do you mean for upstream? Hmmm ... yes I suppose that's doable. None of the changes for this PR require an updated GO-I. We can't easily upstream the GO-I changes since they are to a version that's a bit dated & uses a totally different build system than the current release. I consider the GO-I changes a one-off & we'll deal with updating them when we update GO-I.

[michaelld (Michael Dickens)]

In e6d093a57f6a86b0ace628ff70d6ec79fd3808ee/macports-ports (master):

gobject-introspection: add patch to fix in-build scanner execution

Ref: #58574

[michaelld (Michael Dickens)]

FYI I just merged in part 1: the fixes to GO-I. Next up is part 2: the fixes to Poppler itself.

[michaelld (Michael Dickens)]

In a7bb7220d55fb1c5385b88cce11deb6b1a249ea1/macports-ports (master):

poppler: various fixes for better macOS / MacPorts building

Closes: #58574
Closes: #61316