poppler @0.80.0 build fails with Xcode 11.0 on Mojave

[mndavidoff (Monte Davidoff)]

I received this error when trying to install poppler.

sudo port -vs install poppler
...
clang: warning: no such sysroot directory: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk' [-Wmissing-sysroot]
In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_poppler/poppler/work/build/glib/tmp-introspect5kyi2vdc/Poppler-0.18.c:2:
In file included from /opt/local/include/glib-2.0/glib.h:30:
In file included from /opt/local/include/glib-2.0/glib/galloca.h:32:
/opt/local/include/glib-2.0/glib/gtypes.h:35:10: fatal error: 'time.h' file not found
#include <time.h>
         ^~~~~~~~
1 error generated.

The problem might have been introduced after upgrading to Xcode 11.0.

$ xcodebuild -version
Xcode 11.0

[yan12125 (Chih-Hsuan Yen)]

The cause is that CPython records the sysroot to build (e.g., in /opt/local/Python.framework/Versions/3.7/lib/python3.7/_sysconfigdata_m_darwin_darwin.py). The same issue occurs on all Python packages with C extensions (e.g. py37-curl), and a possible solution might be stripping sysroot paths from _sysconfigdata_*.py.

CC the maintainer of cpython ports.

[jmroot (Joshua Root)]

Deleting from sysconfigdata won't actually fix anything; the correct -isysroot flag still needs to be passed along somehow.

[kencu (Ken)]

Can you pass the SDK in via $SDKROOT ?

I've been working with that lately myself, with some success.

<​https://github.com/macports/macports-base/pull/147>

[jmroot (Joshua Root)]

Looks like g-ir-scanner isn't passing along our CFLAGS.

[yan12125 (Chih-Hsuan Yen)]

As a side note, py37-curl builds fine after [02eac73dfdc362a003adf7148df843b9a5023b31/macports-ports].

Can you pass the SDK in via $SDKROOT ?

I think it's possible. Anyway, as the aforementioned commit can also fix issues, and I prefer simpler approaches :)

[dbevans (David B. Evans)]

The problem can be fixed by properly installing Xcode11 and compatible Command Line Tools with MacPorts 2.6.0 and rebuilding poppler and all dependents from source. Essentially treating the move to Xcode11 on 10.14 as if one were migrating to a newer macOS version. Works for me and doesn't involve trying to muck with the various ports to overcome the broken build environment. Or you can just stay with Xcode 10.3 and switch to Xcode 11 when Catalina is released. See more detailed discussion in duplicate ​https://trac.macports.org/ticket/59091. Note poppler builds fine in the latest Catalina public beta with Xcode 11.

[dbevans (David B. Evans)]

A number of other ports have similar problems when prebuilt ports from Xcode 10 are mixed with Xcode 11. Ports that create perl extensions that link with libperl fail similarly in the mixed environment. Rebuilding say perl5.28 under Xcode 11 allows p5.28 ports to build without problems. The python ports mentioned above are probably another such case.

[mndavidoff (Monte Davidoff)]

As you suggested, I rebuilt python37, perl5.28, and all ports that depend on them with Xcode 11 using port -s. I was then able to install poppler without error.

[fnajm (Farid Najm)]

I started with Xcode 11.0 and a clean macports 2.6.0 (with no ports at all installed), then installed a bunch of ports that I routinely install, but I was using port, not port -s, and everything installed fine except poppler. So, I don't think it's about reusing ports that were previously installed under Xcode 10.

[jmroot (Joshua Root)]

It is, sort of. Since g-ir-scanner isn't using the CFLAGS (including -isysroot${configure.sdkroot}) that MacPorts gives to poppler's configure script, distutils defaults to using the sysroot that was configured when python was built.

Telling everyone not to use binaries isn't much of a solution. There must be some way to make g-ir-scanner use the current sysroot.

[fnajm (Farid Najm)]

This is way over my head, but let me give one further clarification in case it helps you guys. This install of Xcode 11 and macports 2.6.0 was done on a brand-new Mac, where I started from scratch with an upgrade to Mojave 10.4.6 followed by installation of Xcode etc. There was no previous install of Xcode or macports on this Mac. I'm a heavy user of macports and it takes about six hours to install all the ports that I use. In my setup, installing evince failed because it depends on poppler and it was the poppler install that triggered the failure.

[kencu (Ken)]

Replying to yan12125:

As a side note, py37-curl builds fine after [02eac73dfdc362a003adf7148df843b9a5023b31/macports-ports].

I prefer simpler approaches :)

Amen. Perhaps some version of <​​https://github.com/macports/macports-base/pull/147> could be helpful eventually.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

There is a ​pull request that attempts to resolve this issue (at least for the ports that use gobject-introspection).

[dbevans (David B. Evans)]

Replying to MarcusCalhoun-Lopez:

There is a ​pull request that attempts to resolve this issue (at least for the ports that use gobject-introspection).

Since this issue is not limited to gobject-introspection and can be fixed by rebuilding with Xcode11 (specific ports if not all) I hesitate to apply this pull request that only addresses the symptoms of gobject-introspection and not the root cause.

[dbevans (David B. Evans)]

Replying to jmroot:

It is, sort of. Since g-ir-scanner isn't using the CFLAGS (including -isysroot${configure.sdkroot}) that MacPorts gives to poppler's configure script, distutils defaults to using the sysroot that was configured when python was built.

Telling everyone not to use binaries isn't much of a solution. There must be some way to make g-ir-scanner use the current sysroot.

I agree that rebuilding everything isn't really needed but in this case given your analysis, isn't rebuilding python with Xcode11 sufficient to cause gobject-introspection to use the right sysroot? And perhaps other ports that rely on python's configuration to build binaries.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Replying to dbevans:

Replying to MarcusCalhoun-Lopez:

There is a ​pull request that attempts to resolve this issue (at least for the ports that use gobject-introspection).

Since this issue is not limited to gobject-introspection and can be fixed by rebuilding with Xcode11 (specific ports if not all) I hesitate to apply this pull request that only addresses the symptoms of gobject-introspection and not the root cause.

May I humbly suggest that the ​pull request is neither a hack nor even a workaround.
As much as possible, ports should respect MacPorts settings, which includes ${configure.sdkroot}.
Even if we successfully rebuild all of the ports with Xcode 11, I would argue that we should still apply the patch.
The Xcode 11 problem simply revealed an issue with the gobject_introspection PortGroup.

[dbevans (David B. Evans)]

Replying to MarcusCalhoun-Lopez:

Replying to dbevans:

Replying to MarcusCalhoun-Lopez:

There is a ​pull request that attempts to resolve this issue (at least for the ports that use gobject-introspection).

Since this issue is not limited to gobject-introspection and can be fixed by rebuilding with Xcode11 (specific ports if not all) I hesitate to apply this pull request that only addresses the symptoms of gobject-introspection and not the root cause.

May I humbly suggest that the ​pull request is neither a hack nor even a workaround.
As much as possible, ports should respect MacPorts settings, which includes ${configure.sdkroot}.
Even if we successfully rebuild all of the ports with Xcode 11, I would argue that we should still apply the patch.
The Xcode 11 problem simply revealed an issue with the gobject_introspection PortGroup.

I disagree since the root cause seems to be that python passes its build time configuration to ports that build binaries (as does perl). So I hold that rebuilding python against Xcode11 should fix the issue as well as potential issues with other python ports that build binaries. I would prefer a more global solution.

[jmroot (Joshua Root)]

Stripping the sdkroot flags out of python, perl and whatever else they got baked into is also a good idea, but a rather more ambitious project than this simple patch which is itself also correct IMO.

[jmroot (Joshua Root)]

There is <​https://github.com/macports/macports-base/pull/147> which sets SDKROOT globally in the environment but I don't know if it helps in the case where a nonexistent -isysroot path was given on the command line.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Replying to dbevans:

Replying to MarcusCalhoun-Lopez:

Replying to dbevans:

Replying to MarcusCalhoun-Lopez:

There is a ​pull request that attempts to resolve this issue (at least for the ports that use gobject-introspection).

Since this issue is not limited to gobject-introspection and can be fixed by rebuilding with Xcode11 (specific ports if not all) I hesitate to apply this pull request that only addresses the symptoms of gobject-introspection and not the root cause.

May I humbly suggest that the ​pull request is neither a hack nor even a workaround.
As much as possible, ports should respect MacPorts settings, which includes ${configure.sdkroot}.
Even if we successfully rebuild all of the ports with Xcode 11, I would argue that we should still apply the patch.
The Xcode 11 problem simply revealed an issue with the gobject_introspection PortGroup.

I disagree since the root cause seems to be that python passes its build time configuration to ports that build binaries (as does perl). So I hold that rebuilding python against Xcode11 should fix the issue as well as potential issues with other python ports that build binaries. I would prefer a more global solution.

I wold argue that there are two issues that need to be fixed:

1. Prevent Python (and Pearl) from passing build time configurations to other ports (as you said).
2. Have ports respect ${configure.sdkroot}.

I may be missing something, but it seems that this is a general solution.
If we prevent Python from passing -isysroot to its ports, then something has to force g-ir-scanner to use the correct -isysroot.

[dbevans (David B. Evans)]

Replying to MarcusCalhoun-Lopez:

I wold argue that there are two issues that need to be fixed:

1. Prevent Python (and Pearl) from passing build time configurations to other ports (as you said).
2. Have ports respect ${configure.sdkroot}.

I may be missing something, but it seems that this is a general solution.
If we prevent Python from passing -isysroot to its ports, then something has to force g-ir-scanner to use the correct -isysroot.

Unfortunately, perl and python have a good reason for the way they pass the build time configuration. The primary use case that they are interested in is the building of perl, python extensions. Passing the build time parameters ensures that the extensions are built in the same environment as the current libpython, libperl, ensuring compatibility. gobject-introspection is an odd case because it doesn't link with those libraries, but would benefit from rebuilding libpython with Xcode 11 as would the many ports that produce python extensions. I'm not entirely familiar with the internals of how python does this but with perl the mechanism is deeply ingrained into the build philosophy and tools generally.

The real problem here is that Xcode 11 requires a different sysroot than the previous Xcode 10 and is therefore not fully compatible. My political statement for the day is that it is a shame that they didn't retain the old 10.14 sysroot for Xcode 11 on 10.14. Things would have been much easier and, of course, the ultimate solution is to not use Xcode 11 on 10.14 unless there is a compelling need. This is really a Mojave corner case that can be avoided. Catalina should be released before long and start a clean slate for Xcode 11.

At any rate, I think we can both agree that we (or at least I) need to understand the python build mechanism better and refine our solution before merging the gobject-introspection PR. So I proposed to keep the PR open but mark it WIP, not to be merged until we have a better handle on this issue.

[dbevans (David B. Evans)]

By the way, I would be interested if rebuilding python37 against Xcode 11 would solve the reporters issue with poppler and gobject-introspection.

[jmroot (Joshua Root)]

I agree that it would have been better if we still had headers matching the current OS version (/usr/include was a nice place for that), but unfortunately Apple has created a different reality.

Replying to dbevans:

By the way, I would be interested if rebuilding python37 against Xcode 11 would solve the reporters issue with poppler and gobject-introspection.

It would. But we can't control the Xcode version in use by everyone.

[mndavidoff (Monte Davidoff)]

Replying to dbevans:

By the way, I would be interested if rebuilding python37 against Xcode 11 would solve the reporters issue with poppler and gobject-introspection.

Yes, it did. I rebuilt python37, perl5.28, and all ports that depend on them with Xcode 11 using port -s. I was then able to install poppler without error.

[jmroot (Joshua Root)]

Replying to dbevans:

At any rate, I think we can both agree that we (or at least I) need to understand the python build mechanism better and refine our solution before merging the gobject-introspection PR.

I think I understand it well enough. It's purely a matter of the -isysroot flag being recorded along with the other CFLAGS in sysconfig. If you don't use an SDK when building python (like on all pre-Mojave systems), there's no -isysroot in there. Recent python versions even have code to strip out the -isysroot option when building if the referenced SDK doesn't exist, but it doesn't work for reasons that aren't really important.

My view is that all ports, whether python-based or otherwise, should respect configure.sdkroot when building. Building against an SDK that is newer than the target system has its own issues (very little open source code checks for weak-linked symbols for example), but we need to be able to do it.

[dbevans (David B. Evans)]

Replying to jmroot:

Replying to dbevans:

At any rate, I think we can both agree that we (or at least I) need to understand the python build mechanism better and refine our solution before merging the gobject-introspection PR.

I think I understand it well enough. It's purely a matter of the -isysroot flag being recorded along with the other CFLAGS in sysconfig. If you don't use an SDK when building python (like on all pre-Mojave systems), there's no -isysroot in there. Recent python versions even have code to strip out the -isysroot option when building if the referenced SDK doesn't exist, but it doesn't work for reasons that aren't really important.

My view is that all ports, whether python-based or otherwise, should respect configure.sdkroot when building. Building against an SDK that is newer than the target system has its own issues (very little open source code checks for weak-linked symbols for example), but we need to be able to do it.

I'm not against this goal but I would really hate to have to modify a lot of ports individually. Wouldn't it be better to do this in base?

[kencu (Ken)]

Replying to dbevans:

I'm not against this goal but I would really hate to have to modify a lot of ports individually. Wouldn't it be better to do this in base?

we just did [dabf0a7a6043bd3b356f9b99df399159153d8c61/macports-base].

I've been using this for a while now, with great success, and without touching hardly any portfiles (modulo libvpx).

[jmroot (Joshua Root)]

We do add -isysroot${configure.sdkroot} to CFLAGS et al in base. If ports choose not to use them there's not much we can do?

[dbevans (David B. Evans)]

We're getting off topic of the original ticket here. Since the reporter has verified that rebuilding key ports with Xcode 11 resolves the issue of building poppler perhaps we should move to a new venue that directly addresses the global issue. Closing this ticket as works-for-me. Updating summary to reflect the limited scope of this issue (Xcode 11 and Mojave) as well as python's involvement.

[jmroot (Joshua Root)]

Poppler's build failure is very much specific to the gobject_introspection portgroup. It's not really a global issue, unless you mean global to ports that use that portgroup. But we can move discussion to the PR if you'd prefer.

[dbevans (David B. Evans)]

Replying to jmroot:

Poppler's build failure is very much specific to the gobject_introspection portgroup. It's not really a global issue, unless you mean global to ports that use that portgroup. But we can move discussion to the PR if you'd prefer.

OK let's do that. But again gobject-introspection is a subset of the larger issue. All ports that build python or perl extensions are potential problems.

[adulau (Alexandre Dulaunoy)]

I add a similar issue and solved by linking back MacOSX10.14.sdk to MacOSX.sdk in /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs.

ln -s MacOSX.sdk/ MacOSX10.14.sdk

I found old references in some packages to 10.14 instead of the default MacOSX and the includes were not accessible by the compiler.

[jmroot (Joshua Root)]

Don't do that. If you create a link inside Xcode, point it to the actual 10.14 SDK which exists in the Command Line Tools (/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk).

[fnajm (Farid Najm)]

Thanks to adulau and jmroot for these comments. This has totally solved my problem. Compile time has gone down from 8 hours with the -s option to 1 hour without the -s, and everything compiled just fine, including poppler. The remedy was to first uninstall all ports, then apply:

% cd /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs

% sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk MacOSX10.14.sdk

then recompile everything from scratch without the -s.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Has duplicate #59091.
Has duplicate #59151.
Has duplicate #59167.
Has duplicate #59177.
Has duplicate #59196.
Has duplicate #59206.

I think it is fair to say that this ticket should not be closed since people seem to keep running into it, the problem is well understood, and there is a ​fix available.

[murrayE]

I'm seeing the same build failure with Xcode 11.0 (under macOS 10.14.6) with poppler-0.81.0_0. From the log:

:info:build clang: warning: no such sysroot directory: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk' [-Wmissing-sysroot]
:info:build In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_poppler/poppler/work/build/glib/tmp-introspectt6pqsio6/Poppler-0.18.c:2:
:info:build In file included from /opt/local/include/glib-2.0/glib.h:30:
:info:build In file included from /opt/local/include/glib-2.0/glib/galloca.h:32:
:info:build /opt/local/include/glib-2.0/glib/gtypes.h:35:10: fatal error: 'time.h' file not found
:info:build #include <time.h>
:info:build          ^~~~~~~~
:info:build 1 error generated.
:info:build Traceback (most recent call last):
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/unixccompiler.py", line 118, in _compile
:info:build     extra_postargs)

As this is a newer version of poppler, should I file a separate bug report?

[dzogrim (Séв(aѕтιen) L.)]

Hi! I've the same error with

/opt/local/include/glib-2.0/glib/gtypes.h:35:10: fatal error: 'time.h' file not found

under macOS 10.14.6 with Xcode 11.0 (glib2 or glib2-devel…).

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

#59236 is another duplicate.

[jmroot (Joshua Root)]

Reporter of duplicate #59276 is not running Xcode 11, but does not have Xcode installed at all, only the command line tools.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

In 90ac6621ecbbe96695afff100490e75bc9b0a4ec/macports-ports (master):

gobject_introspection: respect configure.sdkroot

Fixes #59078

[Gcenx]

This unfortunately broken the ability to compile +universal on macOS Mojave with the 10.13 SDK & deployment target so it’s not really respecting sdkroot it’s always making sdkroot “/“ in this configuration. Myself and Gverm attached logs to the following closes pull-request ​https://github.com/macports/macports-ports/pull/9850

This also happened to harfbuzz when adding the port group due to needing HarfBuzz-0.0.gi for the latest version of pango.

Reverting the commit gotten +universal builds working again.

[jmroot (Joshua Root)]

Replying to Gcenx:

This unfortunately broken the ability to compile +universal on macOS Mojave with the 10.13 SDK & deployment target so it’s not really respecting sdkroot it’s always making sdkroot “/“ in this configuration. Myself and Gverm attached logs to the following closes pull-request ​https://github.com/macports/macports-ports/pull/9850

The logs attached to that PR all show the correct SDK being used, not /.

[Gcenx]

Replying to jmroot:

Replying to Gcenx:

This unfortunately broken the ability to compile +universal on macOS Mojave with the 10.13 SDK & deployment target so it’s not really respecting sdkroot it’s always making sdkroot “/“ in this configuration. Myself and Gverm attached logs to the following closes pull-request ​https://github.com/macports/macports-ports/pull/9850

The logs attached to that PR all show the correct SDK being used, not /.

All I can think is something is missing from CFLAGS or LDFLAGS after this change, luckily I haven’t needed too many Ports that use this group so overriding them with a local copy with muniversal portgroup is working it’s just not ideal.

[ryandesign (Ryan Carsten Schmidt)]

It actually breaks universal on all platforms. Let's deal with it in a new ticket (#62410) not in this long-closed ticket.