python27 +universal fails to build for ppc+ppc64 on 10.5.8

[barracuda156]

ranlib libpython2.7.a
ranlib: archive member: libpython2.7.a(getbuildinfo.o) fat object file's offset in archive not a multiple of 8) (must be since member is a 64-bit object file)
ranlib: for architecture: ppc64 file: libpython2.7.a(pymath.o) has no symbols
ranlib: for architecture: ppc file: libpython2.7.a(pymath.o) has no symbols
/usr/bin/install -c -d -m 755 Python.framework/Versions/2.7
/usr/bin/gcc-4.2 -o Python.framework/Versions/2.7/Python -arch ppc64 -arch ppc -L/opt/local/libexec/openssl11/lib -L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db48 -arch ppc64 -arch ppc  -dynamiclib \
		-all_load libpython2.7.a -Wl,-single_module \
		-install_name /opt/local/Library/Frameworks/Python.framework/Versions/2.7/Python \
		-compatibility_version 2.7 \
		-current_version 2.7;
/usr/bin/install -c -d -m 755  \
		Python.framework/Versions/2.7/Resources/English.lproj
/usr/bin/install -c -m 644 Mac/Resources/framework/Info.plist \
		Python.framework/Versions/2.7/Resources/Info.plist
ln -fsn 2.7 Python.framework/Versions/Current
ln -fsn Versions/Current/Python Python.framework/Python
ln -fsn Versions/Current/Headers Python.framework/Headers
ln -fsn Versions/Current/Resources Python.framework/Resources
/usr/bin/gcc-4.2 -arch ppc64 -arch ppc -L/opt/local/libexec/openssl11/lib -L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db48 -arch ppc64 -arch ppc -u _PyMac_Error Python.framework/Versions/2.7/Python -o python.exe \
			Modules/python.o \
			 -lintl -ldl  -framework CoreFoundation     
DYLD_FRAMEWORK_PATH=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18 ./python.exe -E -S -m sysconfig --generate-posix-vars ;\
	if test $? -ne 0 ; then \
		echo "generate-posix-vars failed" ; \
		rm -f ./pybuilddir.txt ; \
		exit 1 ; \
	fi
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Lib/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Lib/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Lib/sysconfig.py", line 645, in <module>
    _main()
  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Lib/sysconfig.py", line 633, in _main
    _generate_posix_vars()
  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Lib/sysconfig.py", line 339, in _generate_posix_vars
    pybuilddir = 'build/lib.%s-%s' % (get_platform(), sys.version[:3])
  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Lib/sysconfig.py", line 614, in get_platform
    osname, release, machine)
  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Lib/_osx_support.py", line 498, in get_platform_osx
    "Don't know machine value for archs=%r" % (archs,))
ValueError: Don't know machine value for archs=('ppc', 'ppc64')
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1
make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18'
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18" && /usr/bin/make -j4 -w all 
Exit code: 2
Error: Failed to build python27: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there
is a bug.
36-138:~ arjuna$ port installed python27
The following ports are currently installed:
  python27 @2.7.18_5 (active)

32-bit version builds and installs normally, 64-bit one fail. (Due to that I cannot build Cmake in 64-bit.)

[jmroot (Joshua Root)]

#50821

[kencu (Ken)]

because cmake is just a build tool, it is usually possible to force macports to not need it built universal.

if you tell us here how it is that macports is asking for it universal, (what you are doing exactly), we can probably tell you how to avoid needing it universal.

Building it 64 bit is not usually of any benefit as not many PowerMacs have more than 4G of memory anyway, and even if they did, cmake would be unlikely to need that much.

[barracuda156]

Replying to kencu:

because cmake is just a build tool, it is usually possible to force macports to not need it built universal.

if you tell us here how it is that macports is asking for it universal, (what you are doing exactly), we can probably tell you how to avoid needing it universal.

Building it 64 bit is not usually of any benefit as not many PowerMacs have more than 4G of memory anyway, and even if they did, cmake would be unlikely to need that much.

It will be very helpful to know how to avoid Macports asking every dependency to be build in 64-bit. I am not much concerned if Cmake is 64-bit indeed. But I want to get R built as 64-bit, at least on 10.5.8. But every time I try to use +universal to get ppc and ppc64 for any port, Macports starts trying to build multiple dependencies with this setting. More often than not it fails.

[kencu (Ken)]

So, MacPorts is supposed to handle this properly, but there are some hiccups in MacPorts' logic on this.

MacPorts knows that cmake is a tool that does not need to be the exact architecture being built, because the cmake Portfile tells it so with this line:

​https://github.com/macports/macports-ports/blob/81fa5fc77bfc3090ac0717eb3d8f6c91a74b095e/devel/cmake/Portfile#L23

that says:

installs_libs       no

So if you already have cmake installed as ppc, and then you try to install something that needs cmake as ppc64 or universal, Macports is SUPPOSED to see that line, and use your installed cmake for the build, even though it is not universal.

I find this usually, but does not always, work, for reasons I have no idea why not, and sometimes it is necessary to explicitly tell MacPorts in the Portfile of a given port that you specifically don't care about the architecture of some given other port with a line like this:

​https://github.com/macports/macports-ports/blob/81fa5fc77bfc3090ac0717eb3d8f6c91a74b095e/lang/llvm-9.0/Portfile#L56

which says that you don't care about the architecture of some tool with a line like this:

depends_skip_archcheck-append ld64 subversion

So -- if you know that the architecture of cmake does not matter, you can just install cmake directly:

sudo port -v install cmake

and then if all is working well, MacPorts will not try to rebuild cmake as universal, but if it flubs out for no reason you can understand, you can add:

depends_skip_archcheck-append cmake

in the Portfile that seems to be demanding to rebuild cmake as universal, and then it should no longer try to do that.

There is a different scenario that can occur, however.

Let's say you have no ports installed, and then you try to install some port as +universal that needs cmake.

What happens then is that the +universal part of your request will be automatically passed up the chain to all other ports as explicit variants, so MacPorts will then specifically see that cmake is not yet installed, and helpfully give itself the command to

port install cmake +universal

As far as cmake knows, it can build itself as ppc and ppc64, and so it accepts that, and tries to then build all it's deps as ppc and ppc64.

But then python27 will not build as ppc and ppc64, so the universal build of cmake as ppc and ppc64 fails, and the whole system comes to a halt with a build error of some kind.

The solution, which should be to do something like this:

sudo port clean (everything that was trying to build)
sudo port -v install cmake
sudo port -v install my_needed_port +universal

which should then just build away and leave cmake alone, is not obvious.

This exact same error/issue comes up with many many different build tools, like ninja and gmake and others -- all of which have at one time or another triggered complaints that they won't build universal (see 1000 tickets) when in point of fact, none of them are really needed as universal for the task at hand.

But MacPorts doesn't know if you really want a universal cmake or ninja, or if you are just building something else universal that needs those.

You can see, with a few minutes thinking, that this is kinda hard to sort out for the general case.

So -- short story -- if you first install your build tools WITHOUT using universal, you should be fine for any of them that don't actually need universal code in them.

[kencu (Ken)]

without some further (serious) sorting out, you are stuck installing gcc and clang and llvm as universal, even though most of our gcc ports do not actually install themselves as universal in so doing ("Lies to MacPorts, as Ryan likes to say") and clang/llvm have no need to be built as multiple architectures, being native cross-compilers --- but these things would need to be better sorted out for this to actually work right.

For now, you have to install gcc and clang and llvm as +universal if you want to build anything universal with them.

[barracuda156]

Replying to kencu:

without some further (serious) sorting out, you are stuck installing gcc and clang and llvm as universal, even though most of our gcc ports do not actually install themselves as universal in so doing ("Lies to MacPorts, as Ryan likes to say") and clang/llvm have no need to be built as multiple architectures, being native cross-compilers --- but these things would need to be better sorted out for this to actually work right.

For now, you have to install gcc and clang and llvm as +universal if you want to build anything universal with them.

Thank you for detailed explanation.

So since I do require gcc7 to build R, and you say gcc then has to be universal, I am running into the same problem with python27. I did add a line in a portfile, but when I invoke a call to upgrade gcc7 as universal, python27 starts downloading and fails to build.

I used this command:

sudo port -v upgrade --enforce-variants gcc7 +universal

[kencu (Ken)]

I might try this:

sudo port -f deactivate libgcc libgcc7 gcc7
sudo port -v install gcc7 +universal

you can just reactivate if things go sour.

[barracuda156]

Replying to kencu:

I might try this:

sudo port -f deactivate libgcc libgcc7 gcc7
sudo port -v install gcc7 +universal

you can just reactivate if things go sour.

Thank you, I will try this.

By the way, what is the correct way to install a package built on another machine (or same machine but other OS)? I am asking this since I may use an Intel Mac to build something as universal (like you did with Clang) that fails to build on PPC Mac.

Another reason is that for 10.6 PPC port +universal does nothing, however it is reported that 10.6 PPC can in fact build for ppc64 in command line with no issues: ​https://forums.macrumors.com/threads/darwin10-snow-leopard-on-powerpc-ppc7450-g4-how-to-get-irssi-working-via-ports-from-source-or-mix-of-both-solved.2307200/page-4?post=30709495#post-30709495 That is, the issue is on Macports side, but I cannot ask this to be fixed, as no one here got 10.6 PPC. I can however build on 10.5.8 PPC as universal (whatever does build there) and pass that on to 10.6 PPC, or possibly build on 10.6.8 Intel for 10.6 PPC after installing needed components via XcodeLegacy.

[kencu (Ken)]

when I used the "file" command on /usr/lib/libSystem.dylib while in 10.6 PPC, it showed archs of i386, x86_64, and ppc7400 (or perhaps it was just ppc), but no ppc64.

Is the version of 10.6 PPC you have showing something different?

[kencu (Ken)]

The Xcodes I have used on 10.6 (Intel) already have PPC support in them (ppc7400 at least) as shipped by Apple, and so did not need to use XcodeLegacy to add PPC support.

[kencu (Ken)]

Replying to barracuda156:

Another reason is that for 10.6 PPC port +universal does nothing,

Well, only because you turned off universal building by adding these lines to macports.conf:

build_arch              ppc
universal_archs
buildfromsource         always
cxx_stdlib              libstdc++

If you really want to try universal building, change it to this:

build_arch              ppc
universal_archs         ppc ppc64
buildfromsource         always
cxx_stdlib              libstdc++

[barracuda156]

Replying to kencu:

Replying to barracuda156:

Another reason is that for 10.6 PPC port +universal does nothing,

Well, only because you turned off universal building by adding these lines to macports.conf:

build_arch              ppc
universal_archs
buildfromsource         always
cxx_stdlib              libstdc++

If you really want to try universal building, change it to this:

build_arch              ppc
universal_archs         ppc ppc64
buildfromsource         always
cxx_stdlib              libstdc++

Oh no, I remembered to change it to "ppc ppc64". Something else causes the problem.

[barracuda156]

Replying to kencu:

The Xcodes I have used on 10.6 (Intel) already have PPC support in them (ppc7400 at least) as shipped by Apple, and so did not need to use XcodeLegacy to add PPC support.

Xcode 3.2 does have, Xcode 4.2 does not, at least from what I read online. However it can be easily added to Xcode 4.2 (indeed without XcodeLegacy): ​https://stackoverflow.com/questions/5333490/how-can-we-restore-ppc-ppc64-as-well-as-full-10-4-10-5-sdk-support-to-xcode-4 ​https://superuser.com/questions/290575/mac-os-x-10-6-xcode-4-powerpc-c-compiler

[kencu (Ken)]

well before you spend 5 more seconds on it, see if you have the ppc64 support you are hoping for in /usr/lib/libSystem.B.dylib on your 10.6-for-PPC system; my copy did not, so it is a waste of time:

$ file /usr/lib/libSystem.B.dylib
/usr/lib/libSystem.B.dylib: Mach-O universal binary with 3 architectures: [x86_64:Mach-O 64-bit x86_64 dynamically linked shared library, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL>] [i386] [ppc_7400]

(that is on my 10.6.8 Intel system -- the 10.6 PPC system I spun up in the summer is long gone now, but showed the same).

[kencu (Ken)]

Replying to barracuda156:

Xcode 3.2 does have, Xcode 4.2 does not

I think you're quite right there. I did not notice before that there was a difference in PPC support between the two Xcode versions.

To be honest, I have very rarely tried building anything as PPC on my 10.6.8 Intel system; I think I might have only done it once or twice as a test for some "hello_world.c" program many years ago. So I am no expert in cross compiling to PPC from 10.6.8 Intel, for sure!

I did use the cross compiling more on 10.5 Leopard Intel, when I had dreams of a working llvm/clang on 10.5 PPC. And of course, it works quite reliably there, at least for the Apple gcc compilers. None of the usual MacPorts-installed gcc compilers can cross compile as we install them (although there are some gccs in macports that specifically do cross-compile to other systems). The clang/llvms can cross compile to PPC, but the binaries have an incorrect Darwin PowerPC ABI layout and so are ultimately broken.

You can make your own gcc cross-compiler on Leopard easily, however. I have done that several times.

[barracuda156]

Replying to kencu:

well before you spend 5 more seconds on it, see if you have the ppc64 support you are hoping for in /usr/lib/libSystem.B.dylib on your 10.6-for-PPC system; my copy did not, so it is a waste of time:

$ file /usr/lib/libSystem.B.dylib
/usr/lib/libSystem.B.dylib: Mach-O universal binary with 3 architectures: [x86_64:Mach-O 64-bit x86_64 dynamically linked shared library, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL>] [i386] [ppc_7400]

Thank you very much for pointing to this. Indeed, 10A190 has 3 archs there, while 10.5.8 PPC has 4. So that’s why Macports passes -arch ppc when building anything on 10.6. (Sorry if a silly question, but simply replacing the dylib won’t solve the issue, right?)

16. S. On a positive note, I have made gcc 4.2.1 from 10A222 (build 5626) work on 10A190. Had to replace ld twice and gnumake, since 10A222 had those Intel-only. After that compiler works normally.

[kencu (Ken)]

Replying to barracuda156:

So that’s why Macports passes -arch ppc when building anything on 10.6.

That's not exactly it. MacPorts passes -arch ppc when building on 10.6 for ppc because the compiler supports it and so macports adds the archflags to build CXXFLAGS, LDFLAGS, and CFLAGS during it's port configure setup. If you find a file called "portconfigure.tcl" you will see all the logic and functions used in there.

It knows to pass "ppc" specifically because you set this, in macports.conf:

build_arch              ppc

MacPorts doesn't actually know anything about what is in libSystem.B.dylib at this point.

The person who told you they were successfully building +universal (presuming that meant ppc and ppc64) on 10.6-for-PPC must have been just mistaken, as it would appear to be impossible to do so with no ppc64 in the system anywhere.

(Sorry if a silly question, but simply replacing the dylib won’t solve the issue, right?)

You can't just take the one from 10.5.8 and sub it in to 10.6-for-PPC, if that is what you mean -- the 10.6-for-PPC system is built against symbols in that 10.6 libSystem.B.dylib and many of these are not going to be in 10.5.8 (if they were -- just stay on 10.5.8 :> -- which is what I have been recommending to you anyway, as 10.6-for-PPC strikes me as a total waste of time.)

To actually make ppc64 work on 10.6-for-PPC you would have to go through all the system components (not just libSystem.B.dylib, but every library) find the correct version of the source code (almost impossible) and add a build arch of ppc64 to however they build it originally. This is really going to be hard. These system components are built in different ways, some with Xcode, some with an interesting custom build system used internally in Apple that we can see glimpses of in our apple-gcc-4.2, cctools, ld64, and libmacho ports.

And then once you go actually go through the who-knows-how-many fixes needed to get 64bit PowerPC building, you'd have to start seeing what bugs there are, etc.

I can't see anyone doing this, but .

[barracuda156]

Replying to kencu:

If you find a file called "portconfigure.tcl" you will see all the logic and functions used in there.

This was super-helpful tip! Yes, now I have found a way to make Macports pass -arch ppc64 to compiler, and as you told, it breaks down. The same compiler builds the same port for ppc without issues and tricks, and completely freaks out when ppc64 passed.

(Basically 10.6 SDK had ppc64 lacking in portconfigure.tcl, which is why even setting build_arch ppc64 in macports.conf had no effect.)

If 10.6 SDK is the problem, then cross-compiling on Intel Mac with powerpc64-apple-darwin10 target against 10.6 SDK will be likewise pointless.

I have two things to clarify here:

1. Does lack of proper support of ppc64 in 10.6 mean that it won′t run ppc64 apps properly (provided they are built against 10.5 SDK)?

2. If it will run, then is there a "correct" way to move a port built on 10.5.8 into 10.6 in a way it is recognized by Macports on the latter?

I recall you manually put some port into verified folder to make it install, but here there is no such option, since we blocked binary archives. I may perhaps make a shared local archive as described in Macports guide, however 10.6 won′t pull binaries compiled for 10.5 from there, right? Finally, I know it might work building a port on 10.5.8 (without installing) and then moving portions of it into interrupted build on 10.6 – that is a bad method, I guess, but it helped me with mpich which otherwise persistently failed on 10.6 (however I moved only one library, not the whole thing). Port pkg is useless – I could not make a port installed from a package seen by Macports however tried. This perhaps can be a fallback option – at least for R framework. I could build it as 64-bit on 10.5.8, make mpkg and install into 10.6, outside of Macports (after all, once built, R is self-sufficient).

[kencu (Ken)]

You can't run ppc64 software if the system has no ppc64 support, even if compiled on a system that does. You can't static link the system on Darwin.

And anyway if you're building it on 10.5, you might as well run it there, where ppc64 at least was once-upon-a-time tested.

BTW, my G5 is running ppc64 debian 11, with R, llvm/clang-13, rust, qt5 version 15.something, gcc11, octave, etc

Use that instead.

[barracuda156]

Replying to kencu:

You can't run ppc64 software if the system has no ppc64 support, even if compiled on a system that does. You can't static link the system on Darwin.

Got it, thank you. 10.6+ppc64 topic can be considered closed. (If I run into troubles with ppc64 on 10.5.8, I gonna open new tickets normally.)

And anyway if you're building it on 10.5, you might as well run it there, where ppc64 at least was once-upon-a-time tested.

I will re-install 10.5.8 clean, set Macports to build universal and see if I can reach to R framework being built as 64-bit.

BTW, my G5 is running ppc64 debian 11, with R, llvm/clang-13, rust, qt5 version 15.something, gcc11, octave, etc. Use that instead.

I was considering actually FreeBSD ppc64, however on non-MacOS Unix and any Linux I won′t have other software that I use (think Adobe). And while on MacOS I still have reasonably new gcc (and gcc11 is perhaps realistic), on Linux I gonna have no Adobe at all. Perhaps after I make gcc11 work on PowerPC Mac, I will look into alternative OS options more closely.

[kencu (Ken)]

by default a clean macports on 10.5 PPC wiill have universal archs set to "ppc i386" so you have to change that in macports.conf ... you already know how ... good luck!

[barracuda156]

Replying to kencu:

by default a clean macports on 10.5 PPC wiill have universal archs set to "ppc i386" so you have to change that in macports.conf ... you already know how ... good luck!

Yes, I do. Thank you for detailed explanations, they are very helpful!

[barracuda156]

Replying to jmroot:

#50821

Strangely, the bug has been reported 6 years ago, and no fix.