Possible to add CI and/or buildbots with GCC (on Intel)?

[barracuda156]

Since, unfortunately, there seem to exist a quite common fallacy that macOS == clang (and often implying Xcode clang which came with the OS), we will keep getting ports broken for GCC. It will be great to test that somehow – I do not mean PowerPC here, of course, any convenient Intel set-up will be perfectly fine.

[ryandesign (Ryan Carsten Schmidt)]

Adding automated builds that use gcc on modern macOS on Intel (or Apple Silicon) seems unlikely.

For the buildbot, the answer is a clear no. The buildbot exists to build binaries of ports for each OS in whatever configuration a user would normally encounter on that system. Our buildbot workers for Mac OS X 10.6 and later already do that. If a port indicates it needs gcc or some other compiler, that is used; otherwise MacPorts chooses clang.

As for adding a CI runner to build ports with gcc, there isn't a good way to do that. The obvious way to do it is specifying e.g. configure.compiler=macports-gcc-12 on the command line but that would override any compiler settings in the port so even if a port already indicated via compiler.blacklist *gcc* that it is not compatible with gcc, or forced the use of another compiler by setting configure.compiler macports-clang-16 or compiler.whitelist macports-clang-16, a build with gcc would happen anyway, would fail, and would waste people's time re-investigating a failure that was already known and declared.

Also, you can't mix and match C++ standard libraries. MacPorts uses libc++ with clang++ on Mac OS X 10.6 and later so any port that uses C++ on 10.6 or later has to use libc++. Historically, g++ uses libstdc++, so a port built with g++ on 10.6 or later will fail to link if the port uses any other C++ libraries built by MacPorts using libc++. g++ 10 and later have the ability to use libc++ and support for that was added to MacPorts base last year but there hasn't been a release of MacPorts base containing that change yet and I don't know how well g++ works with libc++.

IIRC there have also been problems where some macOS system headers, in at least some macOS SDK versions, were not compatible with gcc.

If I find a bug report (even outside of MacPorts) where someone is encountering a problem building something on macOS with gcc, I'm likely to advise them to stop trying and to use clang instead. It is the compiler everyone uses on macOS now. Trying to use something different is going to cause lots of problems.

I understand that your goal is to improve the situation for PowerPC Macs where clang is not available and gcc has to be used. But I don't think trying a gcc build on modern Intel macOS is a good proxy for whether something builds on an old PowerPC Mac.

That leaves setting up actual PowerPC Macs buildbot workers which is #66288.

[barracuda156]

Thank you for a detailed reply, Ryan! Re Intel GCC buildbot I did not think implications through: it indeed makes no sense at all and would be a waste of resources. As for CI, provided it does not require ongoing maintenance (I do not know how it works with GH), it still has positive value – exactly because no one tests it otherwise, and bugs may go unnoticed.

I know about libcxx choices in Macports, and I think I may have asked elsewhere to enable an easy way to choose libstdc++ that will be picked by the base (yes, you can set it in macports.conf, but I think everything breaks down in this regard after 10.6). Apple decided to go with Clang, but there is no reason to restrict ourselves to it – I mean, having an option never hurts.

I do not have statistical data on GCC usage on current macOS, but I believe it is used, I have seen mentions on GitHub, and not about PowerPC. But regardless of end-users, it certainly makes sense for testing and development.

Perhaps logic of implementation can be thought through to avoid issues you have pointed at. It should not be hard also to mark GCC checks as suggestive and not obligatory, perhaps. Picking a version of macOS which is known to work fine with GCC headers. I have seen in some GH repos numerous CIs with extensive variety. Some if not many test both with Clang and GCC.

16. S. This is just a discussion, not a request. And I keep it separate from my personal

Interest in PPC. Indeed, for PPC it will be of little help. There also may be complications which I am currently unaware of. If it is a matter of writing a coherent logic and letting GH do its job, that seems worth. If it gonna require ongoing efforts to maintain, then let us forget about it.

[barracuda156]

On 10.6.8 x86_64 setting cxx lib to libstdc++, blacklisting clangs (or otherwise overcoming Macports forcing those on case-by-case basis) and using only GCC works fine. I have two such installations of Macports, where I bootstrap gcc10 and then build gcc12 with it.

As long as universal is not attempted (which does not work correctly with Macports gcc, even for Intel), such set-up is perfectly functional. I do not think I had to do many fixes manually, if any at all.

Unfortunately, support for libstdc++ with Clang (or Intel systems) was broken in Macports recently for a reason unknown to me, which left a number of ports broken and which gonna indeed create complications for CI of some ports. But if GCC CI is not enforced for compliance or even made optional (provided technically possible with GH facilities), that is not a stopper, perhaps.

[ryandesign (Ryan Carsten Schmidt)]

If we wanted to provide a gcc CI worker, what's probably needed is a way to identify those ports that it should build, so that it can skip building those it doesn't need to build. Those it doesn't need to build would include those ports where a gcc compiler is already selected by default and those ports that declare they are incompatible with gcc via blacklist. The port command line may not provide a way to make these determinations but there may be a way to do it with the port API, and a Tcl script could be written to do it, along the lines of the ​existing Tcl scripts used by the buildbot.

Replying to barracuda156:

On 10.6.8 x86_64 setting cxx lib to libstdc++, blacklisting clangs (or otherwise overcoming Macports forcing those on case-by-case basis) and using only GCC works fine.

Yes if you build all ports that way that could work, for those ports that are compatible with gcc. But on your hypothetical gcc CI worker, if we set cxx_stdlib to libstdc++, that means it can't use any of our prebuilt binaries that were built for libc++. MacPorts won't use any binaries if the MacPorts installation's cxx_stdlib setting doesn't match that of the collection of archives, not even for those ports that don't use C++. (MacPorts doesn't keep track of which ports use C++.) That means the CI worker would have to build every port's dependencies from source as well. That is likely to run up against the six hour per-run limit imposed by GitHub Actions. So we either keep cxx_stdlib set to libc++ and see how well g++ copes with libc++, or we have to set up a gcc buildbot worker too to provide libstdc++ binaries, and devise a way to store those separately from the libc++ binaries that go to users. This is a lot of effort to go to to verify a configuration (gcc on modern macOS) that nobody really needs. And even if we provide such binaries, they would probably not go through our CDN and would be served directly from the buildbot web server, using up more of its limited bandwidth.

I'm also still just not sure how useful this will be. I imagine two classes of software: 1. Software written to be cross-platform and probably tested most often with gcc on Linux which (if it builds with clang on macOS) has a good chance of building with gcc on macOS too, and 2. Software written specifically for macOS, probably tested only with clang on macOS, which might fail with gcc but who cares because everybody uses clang (acknowledging, again, that PowerPC users care since they don't have clang available).

It feels to me like this (verifying that everything builds with gcc) is not our problem to solve. ​Something will always be broken somewhere and it's not really MacPorts' mission to fix all software everywhere in all build modes. I suggest we focus on fixing ports in the standard build configurations. There's more than enough work to keep everyone busy fixing implicit function declarations, implicit int problems, and the other usual host of problems people report when building with the default compiler.

[kencu (Ken)]

Replying to barracuda156:

Unfortunately, support for libstdc++ with Clang (or Intel systems) was broken in Macports recently for a reason unknown to me, which left a number of ports broken

I have all of my /opt/bootstrap installations of MacPorts set up to use libstdc++ up to 10.8, and they all continue to work fine. I just updated them.

I think you're talking about the fact that Chris removed the support for macports-libstdc++ from the default macports-clang-* builds recently. This is not as much of an issue as it might first seem, really, as you have to build them anyway if you're using non-default settings, and so you build them with macports-libstdc++ enabled.

As far as I know, no ports have been broken by this change.

[barracuda156]

Replying to ryandesign:

Yes if you build all ports that way that could work, for those ports that are compatible with gcc. But on your hypothetical gcc CI worker, if we set cxx_stdlib to libstdc++, that means it can't use any of our prebuilt binaries that were built for libc++. MacPorts won't use any binaries if the MacPorts installation's cxx_stdlib setting doesn't match that of the collection of archives, not even for those ports that don't use C++.

Ok, that makes sense. I have no idea how usable libc++ is, and the last time it was discussed with Iain and Ken, it appeared that no one has an idea.

It feels to me like this (verifying that everything builds with gcc) is not our problem to solve. ​Something will always be broken somewhere and it's not really MacPorts' mission to fix all software everywhere in all build modes. I suggest we focus on fixing ports in the standard build configurations. There's more than enough work to keep everyone busy fixing implicit function declarations, implicit int problems, and the other usual host of problems people report when building with the default compiler.

I totally agree. As I mentioned above, while I think it makes sense to test with GCC per se, practical meaningfulness depends on costs vs benefits. If costs (in whatever sense) outweigh, it is not economical to do it.

[barracuda156]

Replying to kencu:

Replying to barracuda156:

Unfortunately, support for libstdc++ with Clang (or Intel systems) was broken in Macports recently for a reason unknown to me, which left a number of ports broken

I have all of my /opt/bootstrap installations of MacPorts set up to use libstdc++ up to 10.8, and they all continue to work fine. I just updated them.

I think you're talking about the fact that Chris removed the support for macports-libstdc++ from the default macports-clang-* builds recently. This is not as much of an issue as it might first seem, really, as you have to build them anyway if you're using non-default settings, and so you build them with macports-libstdc++ enabled.

As far as I know, no ports have been broken by this change.

I am not sure which specifically change created an issue, but I believe that, for example, darwinbuild may be broken now on Intel (it is broken on PPC too, but for a totally different reason): ​https://trac.macports.org/ticket/67208 (Need to verify, away from Intel machine right now. It might be working, who knows; last time I tried was a while ago.)

[kencu (Ken)]

darwinbuild has never worked all the years I’ve been in macports….but that has nothing to do with what Chris did to the clangs.