Request buildbot to pre-compile all ports for Ventura (Darwin 22) arm64

[ckyoog (Yu Kou)]

Request buildbot to pre-compile all ports for Ventura (Darwin 22) arm64. The pre-compiled binaries for Ventura arm64 are not available, while the ones for Ventura x86_64 are already available. Does anyone know why buildbot has not started to build binaries for Ventura (Darwin 22) arm64? I am a new user of macports. I am wondering what's the timing usually for macports team decides to build pre-compiled binaries for a new version/architecture.

[jmroot (Joshua Root)]

We decide to build binaries as soon as a new OS exists. Actually doing it is a little more work, which is already in progress. ​https://build.macports.org/builders/ports-13_arm64-builder

[sierkb (Sierk Bornemann)]

Replying to jmroot:

We decide to build binaries as soon as a new OS exists. Actually doing it is a little more work, which is already in progress. ​https://build.macports.org/builders/ports-13_arm64-builder

Wouldn't it be helpful to place a short info note on ​https://trac.macports.org/wiki/VenturaProblems meanwhile, stating that there is a delay providing arm64 packages and naming the cause of the delay (for instance – maybe this might be the case here, I speculate – a Ventura capable Apple Silicon hardware is not yet at hand and connected to be able to build and provide arm64 Ventura (darwin_22.arm64) packages)?

Additional question: as long as *.darwin_22.arm64.tbz2 packages can't be build and provided and so long only *.darwin_22.x86_64.tbz2 packages at the moment, is there a possibility to let MacPorts fetch and install the pre-build packages of *.darwin_21.arm64.tbz2 (Darwin 21 packages) instead meanwhile as temporary workaround without rebuilding them locally, and fetching and installing the Darwin 22 packages replacing these, when Darwin 22 arm64 packages are available?

[ckyoog (Yu Kou)]

Replying to jmroot:

We decide to build binaries as soon as a new OS exists. Actually doing it is a little more work, which is already in progress. ​https://build.macports.org/builders/ports-13_arm64-builder

The build status is showing offline in the link you gave, does that mean the building is not started yet? And yeah, that would great if the causes of the delay can be listed somewhere.

[ryandesign (Ryan Carsten Schmidt)]

I have not begun building archives for Ventura for arm64. I don't have a machine available to do it on. We only have one arm64 machine, a Mac mini loaned to us by Mac Stadium. It boots back and forth between Big Sur and Monterey to make the archives for those OS versions. Each OS version takes about 100GB (including the OS, Xcode, and the successfully-built ports), and at least 20GB should remain free to allow the largest ports to be built. The disk has a capacity of 256GB so there is not room for a third OS environment. I will ask Mac Stadium if we can trade this Mac mini for one with a 512GB disk. If we cannot, then I probably have to stop making archives for Big Sur in order to begin making archives for Ventura.

We do not support installing archives that were built on a different OS version.

[sierkb (Sierk Bornemann)]

Replying to ryandesign:

Ryan, thank you for your explanation and clarification. I don't think it's wrong to say a big thank you at this point for your dedication and commitment in general and also in this special case. My thanks go to you and the whole MacPorts project team, you are doing a great job. Let's hope together that there will be a viable solution to the problem in the not too distant future, by whatever means. I will practice patience and trust you.

[ckyoog (Yu Kou)]

Thank you Ryan. Looks like we need to continue to wait, but the good part is we know what is happening, that makes the issue no more pending. Hopefully it won't be too long, and of course, good luck to you!

[jmroot (Joshua Root)]

Replying to ryandesign:

If we cannot, then I probably have to stop making archives for Big Sur in order to begin making archives for Ventura.

I'd say that's the best tradeoff with our current resources. I guess it would be possible to occasionally bring up the Big Sur builder if all ports are uninstalled from the Monterey one to make room beforehand.

[andreasf (Andreas Fleig)]

Cirrus CI have open-sourced some of their internal tools that allow self-hosting CI with virtualized macOS machines: ​https://tart.run/cirrus-cli/

Given enough disk space, this looks like a great solution to run buildbots for several OS versions on a single machine.

They also provide free macOS CI services for open source projects: ​https://cirrus-ci.org/.

[andreasf (Andreas Fleig)]

Actually, the cirrus cli would allow trading disk space for network traffic. Images can be stored in OCI registries. An image that is currently not needed can simply be deleted from the machine. The next time it is needed, the tool will download it again.

[dgilman (David Gilman)]

Is it possible to make a hardware donation? Anyone want to split the cost of a m1 mac mini?

[jmroot (Joshua Root)]

MacPorts is not part of a registered non-profit, so it wouldn't be tax deductible or anything, but I don't know of any reason why a group of people couldn't give Ryan a Mac.

[kpankonen (Kevin Pankonen)]

Replying to dgilman:

Is it possible to make a hardware donation? Anyone want to split the cost of a m1 mac mini?

I'd be happy to

[ryandesign (Ryan Carsten Schmidt)]

Thank you for your generous offer! However I think we can postpone the acquisition of our own Apple Silicon hardware. Mac Stadium says they are willing to swap our Mac mini for one with more disk space which should give us enough room for macOS 13 and 14 at least.

[fhgwright (Fred Wright)]

Another possibility that's less expensive and probably better anyway is to add an external drive, via something like this:

​https://eshop.macsales.com/shop/external-drives/owc-ministack

Moving heavy build activity to an external drive is beneficial because the internal "SSD" isn't really an SSD at all, but rather some flash chips soldered to the motherboard. Once the flash wears out, it's a really expensive repair.

[ryandesign (Ryan Carsten Schmidt)]

We do not have the ability to add external drives or other hardware to the machine; it is colocated at Mac Stadium. If the SSD wears out, it would be Mac Stadium's problem since they own the machine.

[wyuenho (Jimmy Yuen Ho Wong)]

Why is this taking so long? Is this something that can be solved by money? Is there a opencollective account out there for people to donate to this project? As of now, macports is very unusable on Ventura. Various ports don't build on it and there's no broken build reports generated because of this.

[ryandesign (Ryan Carsten Schmidt)]

It is taking so long because I handle the buildbot system but my interest in dealing with MacPorts is currently very low. I did not check my ticket with Mac Stadium until now to see their latest response.

Mac Stadium has changed their mind and we are not able to get a bigger Mac mini unless we pay the difference in price which seems to be about $20/mo. This seems to be a change from when we originally got the machine. My recollection is that any of the Mac minis was within their free open source allowance, but I selected the smaller 256GB disk rather than the 512GB because at the time I did not anticipate that we would attempt to run more than one OS on it so I could not imagine needing more disk space. Today, they don't even offer a 512GB disk anymore; it's either 256GB or 1TB.

MacPorts is not a legal entity therefore it does not accept donations. I do personally ​accept donations on GitHub Sponsors. I think we should not add another monthly bill when my current MacPorts-related costs are not yet being covered.

While cleaning up the machine in preparation for the now no longer planned decommissioning, I made another attempt to locate any unneeded disk usage and found 18GB wasted due to #60185. Cleaning that up, along with deleting unneeded Xcode simulator caches (whose creation I would love to learn how to prevent in the first place), may give us enough space to add Ventura.

If a port does not build for you, it probably won't build on the buildbot either. File a bug report.

[kencu (Ken)]

I have hundreds and hundreds of ports built on my M1, and have fixed nearly all the Ventura tickets that came up. There are almost none.

The few remaining tickets are either unfixable ( for me anyway) or await known solutions to be committed by the maintainer.

If you find a port that’s broken on Ventura, open a ticket… it’ll get fixed.

The M1s are SO fast the lack of a buildbot hasn’t been much of an issue for me…even the compilers build in way under an hour, most things are nearly instantaneous.

For that ncurses thing that is bugging you, ask Josh nicely to commit the “hacky” fix.

[dm3jan (Janek)]

Seems that now a builder for Ventura/arm64 is running, thanks for your efforts Ryan! :)

[qpanda]

Thank you for getting the builder for Ventura / arm64 up and running, this is great, much appreciated! Hope this can be achieved more quickly for the next macOS version.