Move ports: xnu-headers, libc-headers, CarbonHeaders, etc

[watsodw]

clang-3.3 fails to upgrade with error Availability.h not found.

[cooljeanius (Eric Gallager)]

The CarbonHeaders port contains an Availability.h header; try installing that.

[jeremyhu (Jeremy Huddleston Sequoia)]

Availability.h exists in /usr/include on 10.5+

You're missing your dependencies.

[watsodw]

Availability does exist in my /usr/include, but my setup kept clang from finding it. An install of CarbonHeaders allowed clang to install.

[bernard.j.kelly@…]

Cc Me!

[bernard.j.kelly@…]

This just happened to me on my Mountain Lion (10.8.4) system.

I've just installed CarbonHeaders, and that appears to have fixed the issue, but I'm wondering why this got flagged as "invalid"? Did earlier clang versions not need this header? If an installed port breaks on "upgrade outdated" when nothing was (consciously) changed by the user, isn't that a real problem?

[jeremyhu (Jeremy Huddleston Sequoia)]

See comment:3 for why this is invalid. This is a system header, and if you're missing it, you either deleted it or never installed it.

[bernard.j.kelly@…]

Replying to jeremyhu@…:

See comment #3 for why this is invalid. This is a system header, and if you're missing it, you either deleted it or never installed it.

As for david.w.watson's machine above, Availability.h was always sitting in my /usr/include, but MacPorts (or the clang build process) couldn't find it. What environment variable(s) should it be using to look there? Is it possible that this particular release of clang is relying on an environment variable previous versions didn't look at?

[jeremyhu (Jeremy Huddleston Sequoia)]

Well I'm not hitting this on any of my machines. This is almost certainly a configuration error. If you don't think so, please provide a reduced case showing the bug.

[bernard.j.kelly@…]

Hi Jeremy. I just did the following:

port uninstall CarbonHeaders
port uninstall clang-3.3
port selfupdate
port upgrade outdated
port install clang-3.3 +python27 +assertions

... and hit the same install error. I'll attach the error log, but to give some "reduced case" as you suggest sounds like I'd have to get a clean machine with no other ports installed, which I'm afraid I cannot do. Do you have any suggestion for what I could supply to show the current configuration?

[bernard.j.kelly@…]

clang-3.3 install error on Mountain Lion machine

[mymacports@…]

Cc Me!

[mymacports@…]

same problem here.

clang-3.3 fails to upgrade.

OS X 10.8.4

Availability.h is in /usr/include

[larryv (Lawrence Velázquez)]

Replying to mymacports@…:

same problem here.

clang-3.3 fails to upgrade.

OS X 10.8.4

Availability.h is in /usr/include

Please attach a ​clean main.log.

[jeremyhu (Jeremy Huddleston Sequoia)]

:info:build /opt/local/include/sys/stat.h:75:10: fatal error: 'Availability.h' file not found
:info:build #include <Availability.h>

Remove libc-headers, CarbonHeaders, and all those other ports that conflict with system headers and try again.

Also, please file tickets against whatever port depended on those ports, so they can fix the real issue rather than forcing users to break their system with broken ports that don't match system libraries.

I will not support these configurations.

[jeremyhu (Jeremy Huddleston Sequoia)]

Cc Me!

[mymacports@…]

1. "... and all those other ports that conflict with system headers"
   There are no other problems except the one with clang-3.3
   

2. I do not have libc-headers and/or CarbonHeaders installed.
   

3. 
   

port installed *headers*
The following ports are currently installed:
  cctools-headers @836_0
  cctools-headers @836_1
  cctools-headers @839_0 (active)
  dyld-headers @210.2.3_0 (active)
  libunwind-headers @35.1_0 (active)
  xnu-headers @2050.18.24_0 (active)

there are no ports in the system that depend on dyld-headers and/or libunwind-headers and/or xnu-headers.

any ideas?

[jeremyhu (Jeremy Huddleston Sequoia)]

Replying to mymacports@…:

1. "... and all those other ports that conflict with system headers"
   There are no other problems except the one with clang-3.3
   

As one of the engineers who maintains those projects for OS X, and a MacPorts maintainer who has seen enough tickets from their fallout, I assure you there are.

2. I do not have libc-headers and/or CarbonHeaders installed.
   

Ok, in your case, it's the dyld-headers and xnu-headers. Please nuke those.

[beany_kelly@…]

Replying to jeremyhu@…:

Ok, in your case, it's the dyld-headers and xnu-headers. Please nuke those.

Hi Jeremy. I just did the "clean" step before attempting to reinstall clang-3.3, but with the same result.

BUT ...

Your mention of dyld-headers reminded me of an issue I've had for months with MacPorts. Any new installation or upgrade requires that I execute

sudo port [install|upgrade|blah] ...

The first thing I see after this is the message:

dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/sudo) is setuid or setgid

Could this be related? It's cropped up in OSX-related boards, and appears within some Macports ticket discussions, but doesn't seem to have its own dedicated ticket ...

[larryv (Lawrence Velázquez)]

Replying to beany_kelly@…:

Replying to jeremyhu@…:

Ok, in your case, it's the dyld-headers and xnu-headers. Please nuke those.

Hi Jeremy. I just did the "clean" step before attempting to reinstall clang-3.3, but with the same result.

That’s because you need to deactivate or uninstall those ports entirely.

BUT ...

Your mention of dyld-headers reminded me of an issue I've had for months with MacPorts. Any new installation or upgrade requires that I execute

sudo port [install|upgrade|blah] ...

The first thing I see after this is the message:

dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/sudo) is setuid or setgid

Could this be related? It's cropped up in OSX-related boards, and appears within some Macports ticket discussions, but doesn't seem to have its own dedicated ticket ...

It doesn’t have a ticket because it’s not a problem. It’s a security measure hard-coded into OS X’s dynamic linker, and it has no effect on searching for headers. If you want the warning to go away, remove the DYLD_ variables from your environment; otherwise just deal with the message.

[beany_kelly@…]

Replying to larryv@…:

It doesn’t have a ticket because it’s not a problem. It’s a security measure hard-coded into OS X’s dynamic linker, and it has no effect on searching for headers. If you want the warning to go away, remove the DYLD_ variables from your environment; otherwise just deal with the message.

In fact, I didn't *have* any DYLD_ environment variables set. I *did* have LD_LIBRARY_PATH set, and the warning went away when I unset that. So LD_LIBRARY_PATH was being interpreted as DYLD_LIBRARY_PATH (or DYLD_something_else) for some reason.

Anyway, I agree that it doesn't seem to be related to this ticket.

[larryv (Lawrence Velázquez)]

Replying to beany_kelly@…:

In fact, I didn't *have* any DYLD_ environment variables set. I *did* have LD_LIBRARY_PATH set, and the warning went away when I unset that. So LD_LIBRARY_PATH was being interpreted as DYLD_LIBRARY_PATH (or DYLD_something_else) for some reason.

My fault, I always forget about that. The linker ​ignores LD_LIBRARY_PATH too; it just doesn’t bother to warn you about it.

Anyway, I agree that it doesn't seem to be related to this ticket.

Any results after deactivating the header ports?

[mymacports@…]

I have uninstalled dyld-headers and xnu-headers.
After that clang-3.3 was compiled without errors.
Thanks!

Replying to jeremyhu@…:

Replying to mymacports@…:

1. "... and all those other ports that conflict with system headers"
   There are no other problems except the one with clang-3.3
   

As one of the engineers who maintains those projects for OS X, and a MacPorts maintainer who has seen enough tickets from their fallout, I assure you there are.

2. I do not have libc-headers and/or CarbonHeaders installed.
   

Ok, in your case, it's the dyld-headers and xnu-headers. Please nuke those.

[beany_kelly@…]

Mine compiled fine also! Tried to post that a few hours ago, but the system wasn't responding.

Presumably those header ports *were* needed once upon a time, or they wouldn't have appeared in our installed ports list.

Perhaps running

sudo port uninstall leaves

would have avoided this? I'm running it now, but didn't think of checking for this functionality before I began composing this update.

[jeremyhu (Jeremy Huddleston Sequoia)]

Replying to beany_kelly@…:

Mine compiled fine also! Tried to post that a few hours ago, but the system wasn't responding.

Presumably those header ports *were* needed once upon a time, or they wouldn't have appeared in our installed ports list.

I think I recall seeing some port pulling them in when I did an upgrade months ago. When I noticed that, I fixed that port to not depend on them. It's likely that people's systems got polluted with those ports...

[neverpanic (Clemens Lang)]

$ port echo depends:dyld-headers or depends:xnu-headers or depends:libc-headers or depends:CarbonHeaders
ld64

[ryandesign (Ryan Carsten Schmidt)]

The newly-submitted lsyncd port (#42129) needs xnu-headers. Can we fix xnu-headers to install the correct headers for each OS X version?

[mfeiri]

Ah, this is something I want to fix for quite a while. I'll update the ports next weekend. To avoid compatibility problems related to the *-headers ports, I plan to move these files from ${prefix}/include/ to ${prefix}/SDKs/Darwin${os.major}.sdk/. This way ports can opt-in to use these files through the -I or -isysroot flag.

One open question is to consider forwardporting old platform support (e.g. powerpc) to the current SDK. This would avoid the need to maintain versioned content in these ports. But it might require more effort to validate compatibility than continuing to maintain the versioned ports (I guess upstream dropped maintenance for a reason).

[jeremyhu (Jeremy Huddleston Sequoia)]

The newly-submitted lsyncd port (#42129) needs xnu-headers.

*why*?

Can we fix xnu-headers to install the correct headers for each OS X version?

No, you should remove them. They don't belong in dports.

[mfeiri]

I’ve updated xnu-headers and libc-headers to the latest versions, but I haven’t yet finished the move from ${prefix} to ${prefix}/Developer/SDKs/Darwin${os.major}. I also want to do some more testing, e.g. to find out if the other *-headers ports can/need to be moved as well and if the isysroot flag works as expected. ETA is next weekend.

Moving these ports to an "SDKs" subdirectory will make them entirely opt-in. This way we will never see any involuntary interferences again. And using these headers through isysroot implies using the same precedence rules that some ports (such as clang apparently) seem to rely on for system headers. This would probably fix the problem described in this bugreport. Meanwhile the use of isystem in the upcoming macports 2.3 (#40656) might also solve the issue at a lower level. I plan to investigate that as well.

[mfeiri]

Resolved in r117043 for xnu-headers, libc-headers, and libm-headers. The other system header ports cctools-headers, dyld-headers, and libunwind-header should follow soon, but I've opened a seperate issue #42500 for that. The CarbonHeaders port will be merged into xnu-headers.

It turned out that the isystem flag wouldn't change anything for this particular issue. So following upstream to a "relocatable SDKs" model for opt-in usage seems to be the best solution.

[cooljeanius (Eric Gallager)]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Replying to jeremyhu@…:

The newly-submitted lsyncd port (#42129) needs xnu-headers.

*why*?

The developers of lsyncd have determined that the only interface capable of meeting their needs is /dev/fsevents but Apple considers it internal, so they need the internal fsevents.h header from xnu-headers. See ​https://github.com/axkibe/lsyncd/issues/257

[seanfarley (Sean Farley)]

Replying to ryandesign@…:

Replying to jeremyhu@…:

The newly-submitted lsyncd port (#42129) needs xnu-headers.

*why*?

The developers of lsyncd have determined that the only interface capable of meeting their needs is /dev/fsevents but Apple considers it internal, so they need the internal fsevents.h header from xnu-headers. See ​https://github.com/axkibe/lsyncd/issues/257

Their response is really just a copt out. I posted a ​reply to their reasoning pointing out why.