Discussion re pre-built ports for PPC

[barracuda156]

Ken, I was thinking a few days back about Macports and PPC, and I have few proposals to consider. Initially I thought to write a single comprehensive post, but now I deem it is more practical to separate associated, but not tied to each other issues.

So here is question # 1.

FWIU, we do not have PPC buildbots anymore (correct me if I am wrong). If you or someone can allocate one PowerMac to this end, it would be a superior solution to what I am suggesting. If so, the text below is irrelevant.

Second best solution, IMHO:

I can upload ports that I build on PPC to MacPorts servers, which can be then used for archives. I am willing to do so, since a) that benefits PPC users, which otherwise may need ridiculous amount of time to build certain ports (forget Cmake or GCC, think Guile LOL), and b) marginal cost for myself is nearly zero (time to do uploads, which can be automated).

Anticipating objections/doubts/questions, I will list obligations on my side, which I see necessary, as well as some negative conditions (that is, not from MacPorts side, but from my side: what I cannot or will not do).

What I commit to, if this proposal is followed through:

1. I will only submit ports built with unmodified portfiles from current MacPorts master, no exceptions (unless explicitly agreed to and approved prior to such submission).
2. As long as I have physical access to my PPC hardware, I will update important (or agreed to upon) ports reasonably fast. Unlike the first point, this is not hard commitment at least for technical reasons, but I intend to keep this going.

Then, limitations:

1. Common-sensical, but anyway: is not feasible (or personally desirable either) to build all existing ports, so choice of what is built and submitted is at my discretion. Of course, if something is desirable to have for MacPorts, I am open to such suggestions and will try to fulfill those. (Buildbot automation cannot be implemented.)
2. I am ready to support primarily Leopard (if 10.6 is considered at a later point, that is easy, but let leave this topic for now aside). I can do something for Tiger occasionally, but generally speaking, not particularly interested in it.
3. For reasons you are well aware of, ppc64 and universal builds are often problematic. I will try to build few important ports for ppc+ppc64 (as long as they do build from MacPorts master), but no commitments here.

For sure, there is no implication of any commitment from MacPorts side. You do not have to use anything at all from pre-built ports from my side or may choose to use some and neglect another. Obviously, if I see that nothing at all is used, I shall interpret it as an end to the project. (As I said, marginal cost of try this for my is close to zero: whatever I build locally, I can upload, and do not care if a specific port is used or not.)

The scope of ports considered can be narrowed down (like, only pre-build Guile, GCC and mpich), if that is preferable for MacPorts, or left open (i.e. whatever I build from standard portfiles, I upload).

What do you think?

(P. S. It is fine to drop the idea, if you or anyone thinks it is of no value for MacPorts.)

[ryandesign (Ryan Carsten Schmidt)]

Replying to barracuda156:

If you or someone can allocate one PowerMac to this end, it would be a superior solution to what I am suggesting.

I have recently acquired another broken Power Mac G5. I should be able to swap parts with the other one I have to get a working machine that can then build for Leopard PPC.

[kencu (Ken)]

If Ryan gets his going, that is best.

I have 6 of them, and for a long time set up my own server supplying prebuilt binaries to folks who were interested. I eventually ran out of interest in keeping that up, but all the binaries are still there.

But they are not "clean room" prebuilts, and Ryan's plan is the only one that MacPorts can support.

[barracuda156]

Replying to ryandesign:

Replying to barracuda156:

If you or someone can allocate one PowerMac to this end, it would be a superior solution to what I am suggesting.

I have recently acquired another broken Power Mac G5. I should be able to swap parts with the other one I have to get a working machine that can then build for Leopard PPC.

That would be awesome!

[barracuda156]

Replying to kencu:

If Ryan gets his going, that is best. I have 6 of them, and for a long time set up my own server supplying prebuilt binaries to folks who were interested. I eventually ran out of interest in keeping that up, but all the binaries are still there.

I remember you had a few, which you then used for Linux. (On a side note, does FreeBSD work okay on G5s? It seems to have occasional support in Macports, so I thought it can be tried, but did not get time yet to go through a non-trivial installation process.)

But they are not "clean room" prebuilts, and Ryan's plan is the only one that MacPorts can support.

Yes, Ryan’s plan is the first best.

[barracuda156]

Replying to ryandesign:

Replying to barracuda156:

If you or someone can allocate one PowerMac to this end, it would be a superior solution to what I am suggesting.

I have recently acquired another broken Power Mac G5. I should be able to swap parts with the other one I have to get a working machine that can then build for Leopard PPC.

Please update us if any progress happens in this regard.

[llamallama (Chris Jones)]

Per barracuda156's suggestion in ​https://forums.macrumors.com/threads/unofficial-macports-10-5-binary-repo.2376248/?post=31876813#post-31876810, I'm expressing interest in having binaries available for Leopard (and possibly Tiger?) on PowerPC. I'd also be interested in contributing towards this effort. While I wouldn't be that useful in fixing software issues (I'm more of a Systems Engineer rather than a developer) I would be interested in contributing to the build process.

What can I do?

[kencu (Ken)]

Your best option at this point in time is to set up your own ports sharing system (easy):

wiki:howto/ShareArchives2

and then put them on an accessible web server, and tell everyone interested where it is.

They can then add that location to their binary sources like this:

wiki:howto/ShareArchives2#Step4:Fetcharchives

I have done this for many years, have such a website running, and have shared them amongst a few friends in the past.

However, you unfortunately will then not be in a good position to submit tickets to MacPorts about build issues with your ports. Building ports cleanly and correctly is not a simple process. Unless the ports are built in a clean-room environment such as the buildbots, we have no idea how they might have actually been built.

So -- best thing to do is wait for Ryan. If you are very motivated, and there are enough of you, Ryan would no doubt appreciate some sponsorship through his GitHub sponsorship button to purchase and maintain the PPC hardware he needs to maintain for this. Or a donation of hardware if you have any to spare.

[Zorg-Org (Jean-Baptiste Emanuel Zorg)]

I'd like to throw in a request for 10.5 PPC binaries into the ring as well. I would love to see official builds from the 'clean-room', I have cobbled together my personal collection of port binaries into an unofficial repo but am finding that many of my ports are getting stale already, with new versions of ports and new dependencies abounding.

I see mention of a PowerMac G5, which is no doubt what you'd want for the job. I have a few spare iBook G4s if that is sufficient and of interest though. Not the fastest machines in the world, for certain, but happy to donate if it helps. Else we could perhaps rally the PPC community to the GitHub sponsorship route.

[noderaser]

Yet another fan of PowerPC Macs, casting a vote for more support... It may not seem like a super popular platform these days, but there are dozens of us! Actually, I've started to really see an uptick in PPC interest, must be the nostalgia kicking in.

I have a slow house move pending, eventually I hope to get my small Mac lab back together with some towers that I would be happy to contribute in the name of progress. I might need some help getting them configured, but if hardware is what you need I'll do what I can to help.

[ryandesign (Ryan Carsten Schmidt)]

I have not been able to assemble a working Power Mac G5 because I need to swap the power supply between them, but the heat sink must be removed to reach the screw that holds in the power supply and I don't have a sufficiently long screwdriver.

Another thing you can do to help is to file pull requests that add minimum OS version information to ports that don't have it. For example, if a port requires OS X 10.9 (Darwin 13) or later, the Portfile should contain the line:

platforms {darwin >= 13}

Most ports don't have this information because MacPorts didn't have the capability to represent it until the release of MacPorts 2.8.0 in October 2022 (see #15712). (Some ports use an older method of conveying this information, setting known_fail yes depending on some conditional, which also works. To the extent that the conditional has to do with OS versions only, these should be converted to use the new method as time permits.)

In the absence of this information, all buildbot workers will try to build a port, and it will fail to build on the OS versions on which it doesn't work, and reaching this point of failure will take time (which could have been better spent building something that would actually build). Every time a port is updated, even for a trivial change that would not require a rebuild on OS versions where it was successful, it will trigger a buildbot build on those OS versions where it had previously failed to see if that fixed it. On an Intel Mac it may take minutes to activate all the dependencies and potentially many minutes or even hours to reach the point in a build where it fails. On a PowerPC, it would take much longer. I fear that any PowerPC Mac would not be able to keep up with the volume of commits we currently experience if it is constantly trying and retrying to build ports that we already know have no hope of ever building on such an old machine. By properly indicating minimum OS versions in ports, the buildbot can skip those builds on those OS versions within seconds and move on build a different port.

The same applies to indicating supported architectures. For example, if a port only works on Intel or Apple Silicon processors, then the Portfile should contain the line:

supported_archs arm64 i386 x86_64

[mascguy (Christopher Nielsen)]

Replying to ryandesign:

Another thing you can do to help is to file pull requests that add minimum OS version information to ports that don't have it. For example, if a port requires OS X 10.9 (Darwin 13) or later, the Portfile should contain the line:

platforms {darwin >= 13}

Most ports don't have this information because MacPorts didn't have the capability to represent it until the release of MacPorts 2.8.0 in October 2022 (see #15712). (Some ports use an older method of conveying this information, setting known_fail yes depending on some conditional, which also works. To the extent that the conditional has to do with OS versions only, these should be converted to use the new method as time permits.)

Gosh, that is a much cleaner and better approach than known_fail. And wouldn't have known about that, if I didn't happen to read this discussion.

Perhaps we might want to publicize such additions to all members/maintainers, as it's easy for such enhancements to get lost in the release notes of new releases!

[ryandesign (Ryan Carsten Schmidt)]

That's why I mentioned it here, but Josh did also publicize it on the mailing list at the time:

​https://lists.macports.org/pipermail/macports-dev/2022-October/044765.html

and the guide section on the platforms variable was updated as well:

​https://guide.macports.org/#reference.keywords

[barracuda156]

Replying to ryandesign:

Most ports don't have this information because MacPorts didn't have the capability to represent it until the release of MacPorts 2.8.0 in October 2022 (see #15712). (Some ports use an older method of conveying this information, setting known_fail yes depending on some conditional, which also works. To the extent that the conditional has to do with OS versions only, these should be converted to use the new method as time permits.)

In the absence of this information, all buildbot workers will try to build a port, and it will fail to build on the OS versions on which it doesn't work, and reaching this point of failure will take time (which could have been better spent building something that would actually build). Every time a port is updated, even for a trivial change that would not require a rebuild on OS versions where it was successful, it will trigger a buildbot build on those OS versions where it had previously failed to see if that fixed it. On an Intel Mac it may take minutes to activate all the dependencies and potentially many minutes or even hours to reach the point in a build where it fails. On a PowerPC, it would take much longer. I fear that any PowerPC Mac would not be able to keep up with the volume of commits we currently experience if it is constantly trying and retrying to build ports that we already know have no hope of ever building on such an old machine. By properly indicating minimum OS versions in ports, the buildbot can skip those builds on those OS versions within seconds and move on build a different port.

The same applies to indicating supported architectures. For example, if a port only works on Intel or Apple Silicon processors, then the Portfile should contain the line:

supported_archs arm64 i386 x86_64

This will be very helpful indeed to lessen the workload for a hopefully-to-be-added PowerPC build bot.

I assume that everything requiring Rust, Go, GHC, QT5+ and LLVM has a failure set for PowerPC? (That would be an obvious thing to do.)

16. S. Unfortunately we also have bogus exclusions of PPC or unjustified requirements for Clang-only, as was shown recently, for example, with poco and capnproto ports.

[barracuda156]

And yeah, people will appreciate an ability to fetch pre-built ports: ​https://forums.macrumors.com/threads/leopard-sh-and-tiger-sh.2379622/page-3?post=32322828#post-32322828

Imagine building MPLayer from scratch on a G3…

[barracuda156]

Replying to ryandesign:

Replying to barracuda156:

If you or someone can allocate one PowerMac to this end, it would be a superior solution to what I am suggesting.

I have recently acquired another broken Power Mac G5. I should be able to swap parts with the other one I have to get a working machine that can then build for Leopard PPC.

Ryan, sorry to bother on this, but any chance of setting it for Leopard? It takes long time to build some ports on PowerPC, and consider that I have one of the fastest configs locally. I can imagine how discouraging it is for anyone with G4 hardware. We do not really need to build every buildable port, but at least having fundamental ones pre-built would be a great help. (I mean, for Macports’ users with PowerPC hardware, not for myself personally – for myself I can handle it.)