gcc ports: support compiling multiple archs (universal)

[devernay (Frédéric Devernay)]

Apple's gcc, llvm-gcc and clang have a nice feature, which is taht multiple archs can be specified on the same command-line (e.g. "-arch i386 -arch x86_64"). GNU's gcc, as installed by the gcc ports in MacPorts, doesn't have this feature, and thus most ports (actuall all non-muniversal ports) cannot be built universal using gcc. With the following enhancements, *all* ports could be built universal using a modern GNU gcc (with proper OpenMP support etc.)

Actually, what apple did was to write a simple wrapper that calls gcc twice, and then lipo to merge the two objects. That driver can be compiled with modern gcc versions, and works very well. It's a single C file, called driverdriver.c, and can be found at: ​http://www.opensource.apple.com/source/gcc/gcc-5666.3/driverdriver.c?txt

John Bower manage to modify and build that driver for GNU GCC, as installed by macports: ​http://thecoderslife.blogspot.fr/2015/07/building-with-gcc-46-and-xcode-4.html That blog article is about gcc 4.6, but I tested it with GCC 4.8 (built with +universal) and it works very well. I now have to binaries (gcc-mp and g++-mp) that behave exactly like Apple's GCC.

His source code can be found at: ​https://github.com/jhnbwrs/macportsGCCfixup

He added also building and installing configurations for Xcode, but we don't really need these (although that would be a bonus). The main improvement would be the driver.

[mf2k (Frank Schima)]

Trac requires complete email addresses.

[ryandesign (Ryan Carsten Schmidt)]

Although I could not find a prior MacPorts ticket for this, the idea has been mentioned many times before. And although I would welcome this feature, it's unclear how to use John's contributions to provide it. It seems to be organized as a separate project, rather than a set of patches that could be applied to the existing gcc source code. It also seems as though John's instructions rely on the user moving or renaming the compilers installed by the MacPorts gcc ports.

At least some of his information is also out of date. One of the problems mentioned in the blog post is the lack of support for the -arch flag in gcc 4.6. Support for the -arch flag was added in gcc 4.7.

It's not MacPorts' job to extend other people's projects. If this feature is desired, John or you or someone should work with the developers of gcc to incorporate it into the gcc sources. That way, every Mac user, not just those who get their copy of gcc from MacPorts, would benefit.

He also mentions a way to get Xcode to recognize MacPorts compilers. This too has been previously brought to MacPorts' attention. I would welcome this as well, not just for the benefit of users wishing to use gcc in Xcode for their own projects, but for MacPorts itself; we have some ports that use Xcode to build, but that require either a newer or older version of clang than Xcode provides; there is currently no way to build these projects because there is no way to tell Xcode to use a MacPorts clang compiler.

[devernay (Frédéric Devernay)]

Basically, it's just a *single* C file (driverdriver.c) that

• runs the standard gcc-mp-4.8 (or whichever version it was compiled for) for each arch
• runs lipo to make a fat object

It's as simple as that.

John added many things to the project (such as Xcode templates), but only driverdriver.c (which is from Apple) is useful for macports. John made very few modifications to driverdriver.c

Since it's a single file, it could be compiled and installed by every gcc port, it could be named gcc-mp-driver-4.8 or whatever, and it could be used by macport when using gcc, instead of gcc-mp-4.8.

I'm using it everyday to build universal Qt programs compiled with gcc-mp-4.8 (to get OpenMP support) and it works like a charm.

This would solve many issues with universal ports...

Concerning integration into GCC: Apple did that a long time ago, and it never was incorporated in the gcc sources. The GCC guys are very picky, and I am pretty sure they won't integrate anything that depends on external binaries (namely lipo), or that are specific to a few arch's.

[mojca (Mojca Miklavec)]

Did you actually try to get an (official) answer from GCC?

I would find this extension useful as well, but in any case someone needs to write a "complete patch".

[kencu (Ken)]

gcc6 and newer support the -arch flag natively now (the above driver has been incorporated into gcc).

It would be helpful to see gcc6 and libgcc, and perhaps newer gcc versions, support building cross-platform universal again.

I wonder if this might be as simple as reinstating code that was used in apple-gcc-4.2, that was subsequently disabled in gcc 4.4 to gcc 5 (which did not support the -arch flag).

[kencu (Ken)]

Note -- this native support for -arch is gcc6+ only. The other older gcc's could be ignored now I think.

[mojca (Mojca Miklavec)]

Do you have any references about the new functionality? If I try to build a fat binary, it doesn't seem to work:

> g++-mp-6 -arch i386 -arch x86_64 hello.cpp -o hello
g++-mp-6: warning: x86_64 conflicts with i386 (arch flags ignored)

And since I don't have the compiler built universally, -arch i386 doesn't work either, even though it seems like it could work, but only as a synonym for -m32 rather than allowing fat builds.

> g++-mp-6 -arch i386 hello.cpp -o hello
ld: warning: ignoring file /opt/local/lib/gcc6/libstdc++.dylib, file was built for x86_64 which is not the architecture being linked (i386): /opt/local/lib/gcc6/libstdc++.dylib
ld: warning: ld: warning: ignoring file /opt/local/lib/gcc6/libgcc_ext.10.5.dylib, missing required architecture i386 in file /opt/local/lib/gcc6/libgcc_ext.10.5.dylib (1 slices)ignoring file /opt/local/lib/gcc6/gcc/x86_64-apple-darwin17/6.4.0/libgcc.a, file was built for archive which is not the architecture being linked (i386): /opt/local/lib/gcc6/gcc/x86_64-apple-darwin17/6.4.0/libgcc.a

I do remember however that at some point (much before version 6 though) the addition of the -arch flag "solved" failed compilations with CMake & Fortran. Before they added that flag it was nearly impossible to build Fortran code with CMake and CMake devs never bothered to fix that properly. (If some flags are good for C, they must be good for Fortran as well, right?)

[kencu (Ken)]

The functionality that supports darwin specifics is in this file gcc/config/darwin-driver.c. Here's a URL that links to that source <​https://code.woboq.org/gcc/gcc/config/darwin-driver.c.html>.

But it appears the integrated lipo stage has not been included, so multiple archs can't be spec'd by default in gcc. Perhaps that is a deal-breaker for this to work. If so, too bad. I was hoping it might be possible.

I haven't delved into how the muniversal portgroup works, to know if that might help us here.

Anyway, not worth it if it's too complicated to implement.

[mojca (Mojca Miklavec)]

The idea is to make the need to use muniversal obsolete, it's not clear to me how muniversal itself could help in any way (it's just a smart way to circumvent gcc's limitation). My current understanding is that someone needs to write a full patch implementing support for fat binaries and submit it upstream to gcc developers and wait until they accept it. Based on your post I assumed this has already been done for gcc6, but it doesn't seem so.

It's definitely possible to do it. It's just a matter of someone doing it and someone convincing upstream that this is a super cool feature that everyone desperately needs.

[kencu (Ken)]

Yes, true - I had thought that darwin-driver.c was gcc's version of driverdriver.c, but turns out not so much. Pity.