gcc-devel: install on ARM64

[jpanetta (Julian Panetta)]

Edit: this ticket started as this:

Installing OpenBLAS on an Apple Silicon Mac fails during installation of the icu dependency's universal variant due to #45268--even when requesting the -universal OpenBLAS variant.

Is it possible to force the -universal variant to propagate to all dependencies?

But is really now all about installing gcc on Apple Silicon

[jpanetta (Julian Panetta)]

I tried adding -universal to /opt/local/etc/macports/variants.conf as a workaround, but icu+universal is still attempted. I also tried changing universal_archs in macports.conf to only arm64, but this caused other problems...

[kencu (Ken)]

what do you already have installed as +universal?

port -v installed | grep universal

I think, for today, trying to install +universal on an Apple Silicon Mac will be difficult.

[jpanetta (Julian Panetta)]

$ port -v installed | grep universal
  bzip2 @1.0.8_0+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2020-11-22T18:58:15-0800'
  gettext @0.19.8.1_2+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2020-11-23T14:02:37-0800'
  gperf @3.1_0+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2020-11-23T13:56:40-0800'
  libedit @20191231-3.1_0+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2020-11-22T18:57:29-0800'
  libffi @3.3_1+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2020-11-22T18:58:11-0800'
  libiconv @1.16_1+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2020-11-23T13:57:23-0800'
  libtapi @1000.10.8_1+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2020-11-22T18:56:21-0800'
  libtool @2.4.6_10+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2020-11-22T18:57:43-0800'
  ncurses @6.2_0+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2020-11-22T18:57:09-0800'
  xz @5.2.5_0+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2020-11-23T14:03:01-0800'
  zlib @1.2.11_0+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2020-11-23T14:03:05-0800'

[jpanetta (Julian Panetta)]

I would be happy to install a non-universal builds of everything, but I've not managed to do that. Are you suggesting it'll help to uninstall everything that's built with +universal so far?

[jpanetta (Julian Panetta)]

I reinstalled all of those +universal ports so that now they're listed as -universal. However, when I try to install OpenBLAS now, the +universal variants are still getting installed:

$ sudo port install OpenBLAS
--->  Computing dependencies for OpenBLAS
The following dependencies will be installed:
 bzip2
 gcc10
 gettext
 icu
 ld64
 ld64-latest
 libedit
 libffi
 libgcc
 libgcc10
 libiconv
 libmpc
 libtapi
 libtool
 libxml2
 llvm-9.0
 ncurses
 openssl
 xar
 xz
 zlib
Continue? [Y/n]:
--->  Fetching archive for libtapi
--->  Attempting to fetch libtapi-1000.10.8_1+universal.darwin_20.arm64-x86_64.tbz2 from https://packages.macports.org/libtapi
...

[kencu (Ken)]

well -- yeah, I would think so.

Eventually, almost everything should build universal -- but on 23 Nov 2020, I think you will run into lots of potential headaches building universal. In three months, the situation will no doubt look better.

For example, I have not yet enabled meson to build anything universal with arm64 when using the muniversal portgroup, so until I (or someone) does enable that, lots of universal builds might fail.

We need to figure out what you are trying to install that is starting this cascade. What are you after just now?

I would deactivate everything

sudo port -f deactivate active

and then try to install whatever it is you are after

sudo port -v install MYPORT

and see if +universal gets called in. Hopefully not, but if so, we need to figure out what is doing that and stop it from doing that.

[jpanetta (Julian Panetta)]

Thanks, I will try this. Should I remove -universal from my variants.conf before doing this?

[kencu (Ken)]

If you leave -universal in your variants.conf your life will be easier because nothing should build universal.

However, we will not know what started the cascade, but that is our problem.

So -- up to you -- leave it in for easy life and get on with your work. Take it out to figure out what started the mess.

[jpanetta (Julian Panetta)]

If only it were so easy ;). I left -universal in my variants.conf but +universal ports are still being installed with a sudo port install OpenBLAS, the first of which appears to be bzip2-1.0.8_0+universal.darwin_20.arm64-x86_64.tbz2.

This happens after either sudo port -f deactivate active or sudo port uninstall all. Do you have any suggestions for how I can track down what's causing this?

[jpanetta (Julian Panetta)]

Ok, I think I've narrowed things down somewhat. Installing llvm-9.0 tries to bring in universal variants of its dependencies, while manually installing each dependency gets a non-universal variant. So I guess this means the problem originates in llvm-9.0?

[jpanetta (Julian Panetta)]

Aha! The culprit appears to be the supported_archs commands here:

​https://github.com/macports/macports-ports/blob/b2bc6da4d98143000e7534de4f7a931c60eca3a4/lang/llvm-9.0/Portfile#L271 ​https://github.com/macports/macports-ports/blob/b2bc6da4d98143000e7534de4f7a931c60eca3a4/lang/llvm-9.0/Portfile#L398

I commented out the first and added arm64 to the second, and now +universal is not propagated. I'm trying to build this now with fingers crossed.

[kencu (Ken)]

Yes -- we'll have to clean that all up. LLVM-9.0 might build arm64; it's pretty well written that way. clang-9.0 might also, but compiler_rt in clang-9.0 needs some cleanup to build for arm64.

What is asking for llvm-9.0? ld64 should want the +xcode variant, so not that.

I have llvm-devel just finishing an update now, and if it succeeds I'll push that tonight. That is the most likely thing that will build on arm64 right now, and I guess I'll pull out the supported_archs from that and let-'er-rip and see what happens.

I had llvm-11 / clang-11 all set to go but got into a huge fight (once again) with the GitHub PortGroup and the GitHub PortGroup won, so I gave up on that for now while I figure out how I'm going to defeat it.

That should probably build i386/x86_64/arm64 too.

[jpanetta (Julian Panetta)]

Changes to ld64 Makefile to support ld-530

[jpanetta (Julian Panetta)]

Changes to ld64 portfile to support arm64

[jpanetta (Julian Panetta)]

I think it actually was ld64 asking for llvm-9.0, and it's gcc10 asking for ld64.

llvm-9.0 actually built fine for me after those changes I mentioned above. But then ld64 also was missing arm64, and adding it was significantly trickier (I had to update to ld64-530, since backporting arm64 support to ld64-450 looked too difficult). I was able to get 530 working with the attached hacked-together patches. One surprise is the rebase binary seems to have been dropped.

Walking up the dependency tree, now I'm blocked by gcc10, which just says:

error:fetch gcc10 10.2.0 is not supported on Darwin 20 arm
error:fetch Failed to fetch libgcc10: incompatible OS X version
debug:fetch Error code: NONE
debug:fetch Backtrace: incompatible OS X version

:(

[kencu (Ken)]

Oh dear; you should not be using anything other than ld64 +ld64_xcode on newer systems.

ld64 +ld64_xcode doesn't build or need llvm -- it is just a few symlinks into xcode.

If you presently have ld64 installed with any variant other than ld64 +ld64_xcode please do this:

sudo port -f uninstall ld64
sudo port -v install ld64

and you hopefully will get ld64 +ld64_xcode

if you do not get that, -- oops, we have some work to do (which we might well have to do).

By the way, gcc10 will not build for arm64 at present. We do have a nearly-working fork of gcc-devel that is written by GCC's darwin lead that we are desperately trying to get working on Apple Silicon.

At this moment, it builds neat, on the system, but does not build inside MacPort's infrastructure, so that is where we are with that at present.

[kencu (Ken)]

when it comes to ld64-latest, I update that every once in a while -- the trick is it has to / should build all way back to SnowLeopard (it does at present) so I always spend some time working on that.

The only systems that really need ld64-latest are the ones where ld64_xcode is too old for the current SDKs we want to build against. That is not really happening on any systems right now, but if we want, for example, to build against the MacOSX11.0.sdk on Mojave, we might have an issue there. I'll likely have to puff up ld64 for something like that.

[kencu (Ken)]

I have pushed up a new build of llvm/clang/lldb-devel that -- I hope -- will properly handle the supported_archs issue with respect to Apple Silicon and every other system.

[jpanetta (Julian Panetta)]

Thanks, I wasn't aware of that. sudo port install ld64 does not select the +ld64_xcode variant for me, but rather tries to install ld64-latest, hence my efforts to get that building.

Thanks for the information on gcc10!

[kencu (Ken)]

well, one more thing I need to fix. On Apple Silicon, we need to get this to happen:

$ port info ld64
ld64 @3_3 (devel)
Sub-ports:            ld64-97, ld64-127, ld64-236, ld64-274, ld64-latest,
                      ld64-xcode
Variants:             ld64_127, ld64_236, ld64_274, ld64_97, [+]ld64_xcode,
                      universal

[kencu (Ken)]

This part in the ld64 Portfile is supposed to force the ld64_xcode variant for you:

    # Xcode 11 has a newer ld64 than MacPorts currently supplies, so we use it
    if {[vercmp $xcodeversion 11.0] >= 0} {
        default_variants +ld64_xcode
    }

I wonder how it could be that it is not doing that? I haven't seen that issue on Big Sur intel, which gives me this:

% port info ld64
ld64 @3_3 (devel)
Sub-ports:            ld64-97, ld64-127, ld64-236, ld64-274, ld64-latest,
                      ld64-xcode
Variants:             ld64_127, ld64_236, ld64_274, ld64_97, [+]ld64_xcode,
                      universal

[jpanetta (Julian Panetta)]

Probably because I didn't install XCode (just the command line tools)! $xcodeversion for me is none. Sorry for wasting your time with that--downloading the full XCode now. Interestingly, ld64_xcode does still work fine for me if I force it to install.

[kencu (Ken)]

Well -- that is my fault then, for not setting the portfile for ld64 such that it forces the need for Xcode, or otherwise considering that option.

That is worth knowing. We like those xcodeversion tests for certain things, but they are obviously painfully unreliable...

[kencu (Ken)]

In 9a05eaec189f863713e8f8257fe551c1e336fa98/macports-ports (master):

ld64: default to ld64_xcode with system test

we previously used an xcodeversion test to see how new the
system ld64 was, but this test fails when users have only
installed the CLTs, which we are trying to support.

So... we have to use a system version test instead here.

see: #61636#comment:20

[jpanetta (Julian Panetta)]

Cool, thanks! My Xcode installation finally finished, and I can confirm the original check works on Apple Silicon--but it's nice to know I can remove it if my drive fills up.

So I guess I'm stuck for now until gcc-devel is working. Thanks again for all the help!

[kencu (Ken)]

You seem agile enough: the working gcc is here <​https://github.com/iains/gcc-darwin-arm64> if you want to help out.

First build it from scratch, using MP gmp and mpfr or your own.

Once you get that working, you can use it if you like (couple of symlinks and you can hack your way in).

Then we have to get the gcc-devel Portfile to build it and install it into MacPorts prefixes properly... and that is where we are now.

[jpanetta (Julian Panetta)]

I was able to build that gcc branch using the attached Portfile with only a couple changes from the existing gcc-devel version. The main change was adding depends_build port:texinfo to work around a malloc bug in the system /usr/bin/makeinfo.

Unfortunately now it's failing at what I assume is the post-destroot phase:

:debug:destroot Executing proc-post-org.macports.destroot-destroot-0
:debug:destroot PortGroup select: Installing select files to destroot
:info:destroot xinstall: mkdir /opt/local/var/macports/build/_usr_local_ports_lang_gcc-develarm/gcc-develarm/work/destroot/opt/local/etc/select
:info:destroot xinstall: mkdir /opt/local/var/macports/build/_usr_local_ports_lang_gcc-develarm/gcc-develarm/work/destroot/opt/local/etc/select/gcc
:error:destroot Failed to destroot gcc-develarm: xinstall: Cannot stat: /usr/local/ports/lang/gcc-develarm/files/mp-gcc-develarm, No such file or directory
:debug:destroot Error code: NONE
:debug:destroot Backtrace: xinstall: Cannot stat: /usr/local/ports/lang/gcc-develarm/files/mp-gcc-develarm, No such file or directory
:debug:destroot     while executing
:debug:destroot "$post $targetname"
:error:destroot See /opt/local/var/macports/logs/_usr_local_ports_lang_gcc-develarm/gcc-develarm/main.log for details.

though I can't figure out what could be looking for mp-gcc-develarm (all my generated binaries look like gcc-mp-develarm). Also, I tried inserting print statements at the top of post-destroot that do not seem to run. Is this roughly where you guys were stuck?

[jpanetta (Julian Panetta)]

Portfile for building gcc11 on arm

[kencu (Ken)]

you did well!

Michael -- some progress here.

the file you're missing is a macports created file for our "select" process. It has to have a specific name, and sit in the "files" dir under the Portfile.

See this one for the template <​https://github.com/macports/macports-ports/blob/master/lang/gcc-devel/files/mp-gcc-devel>

looks like you're nearly there!

btw, when tweaking destrooting, you don't have to rebuild all of gcc; just delete the destroot folder in the work directory and run the destroot again until you get it right.

[kencu (Ken)]

Michael has the Apple Silicon system and has been running point on getting that gcc into MP; I suspect you've just given some wind into his sails as he's been building it outside but has not been able to get it to build inside MacPorts up to now...

[kencu (Ken)]

I'm thinking that we might just leave this port as being named gcc-devel, but have it use that github repo when building on arm64. That way we won't wind up with a slight mess to clean up later --we'll see what Michael thinks about that too....

[jpanetta (Julian Panetta)]

Great, thanks, with this file it installs! I should have read the missing file path more carefully...

I'm trying to build and install it again after uninstalling libgcc and gcc and cleaning everything. Hopefully it all works now!

[kencu (Ken)]

the issue you will possibly run into is getting MP to actually use it for builds...there is no logic in the portfiles to spec a port named gcc-develarm so the portfiles won't know how to ask for it, exactly, or default to it.

doing this will help a bit to get you using it

sudo port select gcc gcc-develarm

but that will not be enough for the portfiles to find it.

We'll ask Chris.

[kencu (Ken)]

Chris, our intrepid explorer here has done the deed and installed the arm version of gcc on MacPorts.

Can you help us sort out how we might most easily use it, until gcc-11 comes out?

Thanks!

[cjones051073 (Chris Jones)]

I am not sure creating a new port, 'develarm' is necessary here. Why not just update the existing port gcc-devel to (temporarily) use the new branch ? Once the changes are merged with upstream, and/or gcc11 merged, then gcc-devel can just be switched back to the normal snapshots.

[cjones051073 (Chris Jones)]

I presume the Arm branch builds on both Intel and Arm, right ?

[kencu (Ken)]

agree, see what I said too. Our brand new user here has made very excellent progress!!

Lets help him bring it home.

This branch is not at all tested on Intel.

might or might not work.

[cjones051073 (Chris Jones)]

(Sorry about the mess with the ticket changes, I think we submitted our comments at the same time and something got screwed up)...

OK, then I suggest please create a GitHub PR against gcc-devel, we can then go from there. if it doesn't build on Intel that wouldn't actually be a huge deal breaker, short term, but lets see first.

[cjones051073 (Chris Jones)]

just for info, port select intentionally will not help with port builds. We generally do not want port builds paying any attention to this setting. Plus it won't be set on the buildbots..

[kencu (Ken)]

Julian, request is to just switch gcc-devel to the github branch totally and add your depends_build-append.

And then open a PR.

Chris: Yep :)

[kencu (Ken)]

Chris: how do you make the buildbots use gcc-devel anyway? That's not possible unless we edit the libgcc port and change some other compiler selection, right? I think this is all manual for now....unless you know a way...

[cjones051073 (Chris Jones)]

Yes, other changes will be needed. Firstly, I think it will need adding to the compilers PG

​https://github.com/macports/macports-ports/blob/master/_resources/port1.0/group/compilers-1.0.tcl

Once there, it will be available as a 'macports-gcc-devel' compiler selection. Also, any ports using the compilers PG will automatically have it as a variant.

Something else that could also be considered is adding it to

​https://github.com/macports/macports-ports/blob/master/_resources/port1.0/compilers/gcc_compilers.tcl

probably conditionally, only for Arm. That is what defines the default fallbacks.

[jpanetta (Julian Panetta)]

Everything still builds (slowly) after cleaning, though I do see the following alarming issue:

$ sudo port install gcc-develarm
--->  Computing dependencies for gcc-develarm
The following dependencies will be installed:  libgcc-develarm
Continue? [Y/n]:
--->  Fetching distfiles for libgcc-develarm
--->  Verifying checksums for libgcc-develarm
--->  Extracting libgcc-develarm
--->  Configuring libgcc-develarm
--->  Building libgcc-develarm
--->  Staging libgcc-develarm into destroot
--->  Installing libgcc-develarm @6b589e6cb14fb74eaac99411a64083ff32fada85_0
--->  Activating libgcc-develarm @6b589e6cb14fb74eaac99411a64083ff32fada85_0
--->  Cleaning libgcc-develarm
--->  Fetching distfiles for gcc-develarm
--->  Verifying checksums for gcc-develarm
--->  Extracting gcc-develarm
--->  Configuring gcc-develarm
--->  Building gcc-develarm
--->  Staging gcc-develarm into destroot
--->  Installing gcc-develarm @6b589e6cb14fb74eaac99411a64083ff32fada85_0
--->  Activating gcc-develarm @6b589e6cb14fb74eaac99411a64083ff32fada85_0
--->  Cleaning gcc-develarm
--->  Scanning binaries for linking errors
--->  Found 3 broken files, matching files to ports
--->  Found 1 broken port, determining rebuild order
You can always run 'port rev-upgrade' again to fix errors.
The following ports will be rebuilt: libgcc-develarm @6b589e6cb14fb74eaac99411a64083ff32fada85
Continue? [Y/n]:

I should have some time tomorrow to apply these changes to gcc-devel (where hopefully the "broken port" complaint disappears) and submit a pull request.

[cjones051073 (Chris Jones)]

A few comments

• Please do not use the git commit hash as the macports port version. These cannot be used correctly, as e.g. they do not increase monotonically. You will need to mimic what is already used there, a date based on the snapshot used, such that things smoothly migrate from the current port to the new repo (and then back again when we return to using the snapshots).

• The rev-upgrade rebuild errors above could indicate a real issue with the port build, if something is not being linked correctly etc... Please try using the -v verbose port option to see more details.

[cjones051073 (Chris Jones)]

e.g. the current gcc-devel version is

Oberon ~/Projects/MacPorts/ports > port info gcc-devel
gcc-devel @11-20201025 (lang)

so to mimic that please use a version similar, e.g. 11-20201125 (for commit hash of today, for instance).

[jpanetta (Julian Panetta)]

Hmm, I thought I'd done that in my Portfile, so I'm not sure why the commit hash is showing up in port info gcc-develarm for me:

name                gcc-develarm
version             11-20201122

$ port info gcc-develarm
gcc-develarm @6b589e6cb14fb74eaac99411a64083ff32fada85 (lang)

I'll look into the rev-upgrade issue tomorrow.

[cjones051073 (Chris Jones)]

I suspect what is happening when you configure the GitHub repo in the port file its over-writing the macports version. You have to be care to do things in the right order in cases like this.

[cjones051073 (Chris Jones)]

In any case, please first migrate the changes to the current gcc-devel port, then submit a PR, as then its much easier to discuss specific changes there than in these trac tickets.

One other request - when you do change the gcc-devel port please retain, commented out etc., the original settings, so that it is easier to revert back to using the snapshots again in the future.

[kencu (Ken)]

Julian, leave the rest for us...you're not responsible for learning all this MP gobblygook.

You've done plenty enough showing that it builds!

We'll take care of all the rest; it won't take us five minutes, and we've done it 1000 times before.

[kencu (Ken)]

Chris -perhaps we don't really want the buildbots building dozens of ports with a devel version anyway.

I hope everyone realizes this is one repo, not upstreamed yet, an entire new ABI added, certain to be bugs still, has had almost no use in the wild at all, etc, etc, etc.

Don't plan the next SPACEX launch using this!!!!!

[cjones051073 (Chris Jones)]

Replying to kencu:

Chris -perhaps we don't really want the buildbots building dozens of ports with a devel version anyway.

I have no problem with that, to be honest. Its what the devel port is there for, to see how a future not yet released version works on the various platforms. As long as its not updated too frequently I don't see the issue.

Plus if we want the arm buildbots to start using this, to build ports that require a fortran compiler, for instance, then at least that Buildbots will need to build it.

I hope everyone realizes this is one repo, not upstreamed yet, an entire new ABI added, certain to be bugs still, has had almost no use in the wild at all, etc, etc, etc.

Don't plan the next SPACEX launch using this!!!!!

Well, its probably a bit less stable that the normal snapshots, but honestly I would lay the same against the current versions. No one should be using the gcc-devel port for anything serious, regardless of the origin of the snapshot it uses.

[jpanetta (Julian Panetta)]

Okay, I created my pull request: ​https://github.com/macports/macports-ports/pull/9243/

The version issue was indeed just a problem of where the version line appeared in the file. Also, the broken port issue was because libgcc_s.1.dylib is apparently now libgcc_s.2.dylib and wasn't being installed; this is fixed now, and everything seems to work on my machine!

BTW, if I'm not mistaken, it appears to build libgcc twice (once when installing the libgcc-devel package, and again when building gcc-devel). Is this expected behavior? It means installing gcc-devel takes quite some time--even on the M1 :(

[jpanetta (Julian Panetta)]

For what it's worth, this change to compilers-1.0.tcl was enough to get me building OpenBLAS using gcc-devel (my original goal before going down this rabbit hole):

​https://github.com/jpanetta/macports-ports/commit/c92eade99239d8e23cbba770e0bf99ab78de8594

[cjones051073 (Chris Jones)]

Changes committed in ​https://github.com/macports/macports-ports/commit/9098f3dcc378904a7f771c48abee230bc6187e2b