go @1.13: build fails on 10.9 and less

[kencu (Ken)]

I saw this on my 10.7 system this morning, and <​https://ports.macports.org/port/go/builds> shows it affects 10.9 and less.

The special tweaks put in the Portfile to use legacysupport to provide the missing symbols to go will need to be looked at and updated.

[kencu (Ken)]

has duplicate 58937

[Ionic (Mihai Moldovan)]

Have you tried turning it off and on again?

Err, no, seriously, I did upstream the pure LDFLAGS patch and it was accepted for 1.13 back then: ​https://github.com/golang/go/commit/512b3c63b7472d2baab881de4dbcbd0ab8e447ab

So the first patch can definitely be dropped.

The second one needs to be rebased until we have a proper 2.6.0 base release. Limitations in the environment variables parser and such...

[ryandesign (Ryan Carsten Schmidt)]

2.6.0 has been released. Does that help move this ticket along?

[Ionic (Mihai Moldovan)]

I believe so, yes.

We can just drop the additional patch and the legacy env setting stuff if all users are expected to upgrade to the new release right after availability and ports will only be parsed by the new base version.

I'll get working on that after... updating my system.

[Ionic (Mihai Moldovan)]

Just dropping the patches (and the old env setting) doesn't work.

The new issue is go adding $INODE64 to the _fdopendir symbol when the arch is amd64 (== x86_64) and libMacportsLegacySupport doesn't have such a function - only fdopendir. We'll really have to fix the dependency for this to work correctly.

The weird part is that I don't get why go is adding the $INODE64 suffix in the first place. This symbol is used by Apple to disambiguate between struct stat and struct stat64. fdopendir first appeared in 10.10 and doesn't even take a struct stat parameter (but does likely use it internally). I guess they could have just said that they only support the new 64-bit struct for the new function, but that would have created other problems, I guess.

Looking at a 10.11 box, I see that /usr/lib/system/libsystem_c.dylib really includes both symbols:

0000000000081f03 T _fdopendir
000000000002c5ee T _fdopendir$INODE64

So go did the right thing here (and was broken in earlier versions that only used the 32-bit-struct version).

So, to sum it up, before I can push any changes to go itself, I'll have to get the legacy-support port into working condition first.

[kencu (Ken)]

and another as well perhaps <​https://github.com/scottlamb/libc/commit/79dfefa523ab417f398cf56f12f4ba084e7f3237>

$ nm -arch x86_64 /usr/lib/system/libsystem_c.dylib | grep fdopendir
000000000007ea6d T _fdopendir
000000000002ba97 T _fdopendir$INODE64

$ nm -arch i386 /usr/lib/system/libsystem_c.dylib | grep fdopendir
00082d1e T _fdopendir
0002b528 T _fdopendir$INODE64$UNIX2003
00082d1e T _fdopendir$UNIX2003

[Ionic (Mihai Moldovan)]

Oh, right, on 32-bit arches we'll also need the $UNIX2003 suffix, including combinations... meh...

I wonder how to do this correctly, though.

We'd probably need to use the public (if it can even be considered public...) __DARWIN_ALIAS_I macro for our fdopendir definition, but that's only one part of what needs to be changed. We also basically need to compile the same source code twice (or thrice? or four times?) based on the architecture with different macro values set and eventually combine the individual object files into one archive.

N.B.: I fear fdopendir is not the only symbol affected by this, a quick look around revealed that the fstat function is also only included once in libM(p)LS - WITH the $INODE64 suffix on my 10.9 machine.

[Ionic (Mihai Moldovan)]

%  clang -dumpmachine
x86_64-apple-darwin13.4.0
%  clang -m32 -dumpmachine
i386-apple-darwin13.4.0
%  /opt/local/bin/gcc-mp-8 -dumpmachine
x86_64-apple-darwin13
%  /opt/local/bin/gcc-mp-8 -m32 -dumpmachine
x86_64-apple-darwin13

Looks like finding out what target architecture will be generated will be difficult with GCC...

[Ionic (Mihai Moldovan)]

I started writing a shell script getting the target architecture for a given compiler and flags combination, but only realized later that this won't work.

The legacy-support port is not using the muniversal PG but rather requires a compiler supporting -arch flags, meaning that the compiler will do the grunt work and automatically compile for and merge different architectures.

That's... making things difficult on so many levels. We can't build stuff individually for specific arches (like you've seen for x86_64 vs. i386) unless we split the arch flags up and then compile and merge manually.

Not using a configuration framework like autotools makes this even more difficult, because we'll have to do with the Makefile and potentially other shell scripts called from there.

[kencu (Ken)]

I don't think there would be any issue with you adding the muniversal PG to your own test build, for some specific work. Or even building it yourself as one arch or the other using supported_arches in the portfile to force it this way and that way for testing.

But I will admit to trying to keep the legacy support project from getting super over-engineered and collapsing under it's own weight :>

Sometimes, I think it's already getting close to that! There are so many nested include files flying around, and #ifdef blockers, that you can quite easily lose track of what the H*LL is going on, especially with some of that function overriding business...

[Ionic (Mihai Moldovan)]

I'll just try to go the less correct way and build 3 objects for all arches: a stat(32)-based one and two stat(64)-based ones, one of those with strict UNIX03 conformance and the other one without.

Might work, but we'll have more variants than Apple does. Doing it "correctly" is crazy complicated though, even with the muniversal PG.

[Ionic (Mihai Moldovan)]

I've implemented a proof-of-concept in ​https://github.com/macports/macports-legacy-support/compare/master...Ionic:bugfix/fdopendir-variations

The only "proof" I was able to come up with so far is that my original plan doesn't work out:

:info:build Undefined symbols for architecture x86_64:
:info:build   "_close$UNIX2003", referenced from:
:info:build       _fdopendir$INODE64$UNIX2003 in fdopendir.dl.o
:info:build   "_open$UNIX2003", referenced from:
:info:build       _fdopendir$INODE64$UNIX2003 in fdopendir.dl.o
:info:build   "_opendir$INODE64$UNIX2003", referenced from:
:info:build       _fdopendir$INODE64$UNIX2003 in fdopendir.dl.o

This outcome makes sense. The x86_64 architecture does not have any functions with the $UNIX2003 tag. If I understand all of that correctly, the x86_64 architecture was directly UNIX-certified and thus doesn't have any $UNIX2003 functions. The i386 architecture on the other hand wasn't certified and hence needs compatibility symbols that implement the functionality as mandated by SUS, since the "default implementation" is an Apple-specific, non-SUS-compliant one.

The good part is that I can just filter out variants for architectures that don't support them.

Doing that next.

[Ionic (Mihai Moldovan)]

Good, that seems to work:

%  nm -arch x86_64 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep -i fdopendir
0000000000002aa5 T _fdopendir
0000000000002bb1 T _fdopendir$INODE64
%  nm -arch i386 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep -i fdopendir  
00002a55 T _fdopendir
00002c6f T _fdopendir$INODE64$UNIX2003
00002b62 T _fdopendir$UNIX2003

This stuff still seems very fragile...

Ken, you do have access to ppc machines, right?

I'd like to see the output of

% nm -arch ppc /usr/lib/system/libsystem_c.dylib | grep opendir
% nm -arch ppc64 /usr/lib/system/libsystem_c.dylib | grep opendir

on a 10.5, 10.6, and, if possible, also a 10.4 machine.

I need to know how to tweak that stuff for ppc and ppc64.

10.4 is interesting because the first UNIX-certified version was 10.5 as far as I remember, so I suspect that 10.4 won't have $UNIX2003-postfixed symbols. In that case, I'd like to make the fdopendir implementation 10.5+ only.

[kencu (Ken)]

slightly different system locations, but here they come:

TigerPPC:

$ uname -a
Darwin tigerg5.local 8.11.0 Darwin Kernel Version 8.11.0: Wed Oct 10 18:26:00 PDT 2007; root:xnu-792.24.17~1/RELEASE_PPC Power Macintosh powerpc

$ nm -arch ppc /usr/lib/libsystem.dylib | grep opendir
/usr/lib/libsystem.dylib(opendir-UNIX03.So):
900955bc T ___opendir2$UNIX2003
90095990 T _opendir$UNIX2003
         U _opendir
         U ___opendir2
         U _opendir
         U _opendir
         U _opendir
/usr/lib/libsystem.dylib(opendir.So):
9003e480 T ___opendir2
900195f4 T _opendir
         U _opendir

$ nm -arch ppc64 /usr/lib/libsystem.dylib | grep opendir
00000000000b5f30 T ___opendir2
00000000001c5fe0 s ___opendir2$lazy_ptr
00000000001980a0 s ___opendir2$stub
0000000000032708 t _g_opendir
00000000000b63b4 T _opendir
00000000001c5e90 s _opendir$lazy_ptr
0000000000197b60 s _opendir$stub

Leopard PPC:

$ uname -a
Darwin LeopardG5.local 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:57:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_PPC Power Macintosh

$ nm -arch ppc /usr/lib/libsystem.dylib | grep opendir
000a26b0 T ___opendir2
00082dd0 T ___opendir2$INODE64$UNIX2003
0000dc9c T ___opendir2$UNIX2003
0009f1dc T _opendir
00082dc8 T _opendir$INODE64$UNIX2003
0000dc94 T _opendir$UNIX2003

$ nm -arch ppc64 /usr/lib/libsystem.dylib | grep opendir
00000000000a6f14 T ___opendir2
0000000000117074 T ___opendir2$INODE64
00000000000a410c T _opendir
00000000001174ac T _opendir$INODE64

SnowLeopard (Intel):

$ uname -a
Darwin KensMacBookPro.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386

$ nm -arch ppc /usr/lib/libsystem.dylib | grep opendir
/opt/local/libexec/llvm-8.0/bin/llvm-nm: error: file: /usr/lib/libsystem.dylib does not contain architecture: ppc.

$ nm -arch ppc64 /usr/lib/libsystem.dylib | grep opendir
/opt/local/libexec/llvm-8.0/bin/llvm-nm: error: file: /usr/lib/libsystem.dylib does not contain architecture: ppc64.

$ nm -arch i386 /usr/lib/libsystem.dylib | grep opendir
000ac881 T ___opendir2
0000b14a T ___opendir2$INODE64$UNIX2003
000f248e T ___opendir2$UNIX2003
000acd39 T _opendir
0000b12f T _opendir$INODE64$UNIX2003
000f2946 T _opendir$UNIX2003

$ nm -arch x86_64 /usr/lib/libsystem.dylib | grep opendir
00000000000d81aa T ___opendir2
000000000000ab69 T ___opendir2$INODE64
00000000000d85c9 T _opendir
000000000000ab5a T _opendir$INODE64

[kencu (Ken)]

No 64-bit INODES on Tiger... unfortunately.

[Ionic (Mihai Moldovan)]

Great, thanks!

That means that I can treat ppc as an equivalent to i386 and ppc64 as an equivalent to x86_64.

Supporting 10.4 on the other hand will get complicated. Do you think it's worth the effort?

I could pass down the build host's ${os.major} (or something like the deployment target, though I'd like to avoid that one because it's not just a plain number) and use that to achieve 10.4-compatibility. Would make the code a bit more complex, but not by much. For even older systems, building will likely fail (for instance if they don't even have the $UNIX2003 stuff).

I assume that i386 10.4 also only has the "normal" and $UNIX2003 symbols?

[Ionic (Mihai Moldovan)]

I've implemented overrides via the Darwin major version now anyway.

Also fixed up the stuff for thin object files, fixed a few mistakes and split best_fchdir out into its own compilation unit so that it will only be generated once (since we really only need it once anyway), saving space.

I think I'm about happy with that implementation now.

On 10.9 directly, +universal:

%  nm -arch i386 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir 
00002ac1 T _fdopendir
00002c7f T _fdopendir$INODE64$UNIX2003
00002ba0 T _fdopendir$UNIX2003
%  nm -arch x86_64 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir
0000000000002ac2 T _fdopendir
0000000000002bab T _fdopendir$INODE64

On 10.9 directly, -universal:

%  nm -arch i386 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir
error: nm: file: /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_devel_legacy-support/legacy-support/work/destroot/opt/local/lib/libMacportsLegacySupport.dylib does not contain architecture: i386
%  nm -arch x86_64 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir
0000000000002ac2 T _fdopendir
0000000000002bab T _fdopendir$INODE64

Faking 10.4, +universal:

%  nm -arch i386 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir
00002bc0 T _fdopendir
00002c9f T _fdopendir$UNIX2003
%  nm -arch x86_64 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir
0000000000002ac2 T _fdopendir
0000000000002bab T _fdopendir$INODE64

Faking 10.4, -universal:

%  nm -arch i386 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir
error: nm: file: /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_devel_legacy-support/legacy-support/work/destroot/opt/local/lib/libMacportsLegacySupport.dylib does not contain architecture: i386
%  nm -arch x86_64 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir
0000000000002ac2 T _fdopendir
0000000000002bab T _fdopendir$INODE64

I naturally am not able to test anything ppc-related, but I believe it'll likewise work.

Does this look sane? Especially what is being faked for i386 on 10.4?

[kencu (Ken)]

Thanks, beautiful work. I wish I know how to really test it myself :>

10.4 doesn't support $INODE64, sadly. only has 32bit INODES.

How can I see this to test if with you?

[Ionic (Mihai Moldovan)]

Apply this and build it as usual:

​https://git.ionic.de/macports-overlay.git/patch/4911569912ca70a1065158e745440c8e43eb9d77?hp=1c2e7a6117ef6f8b45f27e7a4449fc62028d271c

I haven't bumped up the revision in my repository, mostly because I guess we need a proper release with that code anyway.

Before I create the PR though, I'd like to know that what is being generated matches the symbols of opendir on i386 10.4/Tiger. I guess that it will, though.

[kencu (Ken)]

Chris, please help me take a look at this. It's deep deep deep in Makefile-land, and we don't want to get this wrong.

[Ionic (Mihai Moldovan)]

Well, not autotools, since the project doesn't use autotools. It's purely Makefile and shell, really. :)

The only actual problem I have with that patchset is that the legacy-support port will depend upon cctools from now on. I hope that this won't create dependency loops (because some dependency of cctools depends on legacy-support on older platforms), but there's no good way to get rid of the cctools dependency. The lipo tool, as installed by Xcode or CLT, might be too old to support flags such as -archs (it certainly is on 10.9, so definitely the same on even older platforms), which is needed to get a list of architectures in an object file.

[kencu (Ken)]

I thought lipo went back to forever... The lipo on Tiger works...

The /usr/bin/lipo on this 10.6.8 system I'm on supports -arch...

$ /usr/bin/lipo
/usr/bin/lipo: one of -create, -thin <arch_type>, -extract <arch_type>, -remove <arch_type>, -replace <arch_type> <file_name>, -verify_arch <arch_type> ... , -info or -detailed_info must be specified
/usr/bin/lipo: Usage: /usr/bin/lipo [input_file] ... [-arch <arch_type> input_file] ... [-info] [-detailed_info] [-output output_file] [-create] [-arch_blank <arch_type>] [-thin <arch_type>] [-remove <arch_type>] ... [-extract <arch_type>] ... [-extract_family <arch_type>] ... [-verify_arch <arch_type> ...] [-replace <arch_type> <file_name>] ...

[Ionic (Mihai Moldovan)]

-archs wasn't a typo. ;)

[kencu (Ken)]

BTW, there is no particular reason that the legacysupport port could not use muniversal.

Adding a dep on cctools means llvm-something as well, in most cases, and that could get very messy indeed.

[Ionic (Mihai Moldovan)]

That was my thinking as well, but I'd hope that the bootstrapping llvm/clang and the later default llvm/clang versions won't ever need the legacy-support port.

Using the muniversal PG would be possible, but doesn't really help a lot.

I'm using lipo -archs to get a list of architectures in an object file in a stable(!), scripting-compatible format and I need that information in any case in order to filter out variations that are not suitable for a given architecture (as you've seen, different architectures on the same OS have different symbols). I don't know of any other tool that could do that in a sane and stable version.

Then again, I could rewrite this to make the Makefile take another parameter for the architecture and use that as an override (including checking for non-fatness of object files) instead of relying on lipo (and hence the cctools ports) and use the muniversal PG for MacPorts. That would leave the rather cool auto-detection and fat-file feature using lipo around for people that don't want to build this library as part of MacPorts and still enable us to drop the cctools dependency again?

[Ionic (Mihai Moldovan)]

I've reworked this stuff to use the muniversal PG and not depend upon a newer version of lipo/cctools.

Portfile: ​https://git.ionic.de/macports-overlay.git/blob_plain/1fdddaec503270e0272e4e16da44581be628d7bd:/devel/legacy-support/Portfile

Patch file: ​https://git.ionic.de/macports-overlay.git/blob_plain/1fdddaec503270e0272e4e16da44581be628d7bd:/devel/legacy-support/files/patch-fdopendir-variations.diff

[Ionic (Mihai Moldovan)]

Ping?

[kencu (Ken)]

I thank you for your efforts. It'll take me a bit to catch up to you on this :> I'll start with a branch and see if I can get it building and testing.

[kencu (Ken)]

deleted.

[Mihai Moldovan <ionic@…>]

In bcfdbd30124a6a6cbfa3fd1afc2da704f6d32b92/macports-ports (master):

lang/go: remove custom patches.

The BOOT_GO_LDFLAGS patch was merged upstream, so it's obsolete here.

The patch working around a base env issue via GO_EXT_LDFLAGS can also be
dropped since MP base 2.6.0 has been released.

The port currently needs the legacy-support-devel port to correctly
build on legacy platforms. Once the necessary changes are fully
released, the "normal" legacy-support port will work as well.

Fixes: #58935

[ryandesign (Ryan Carsten Schmidt)]

Reopening since comment:ticket:60221:1 says the issue this ticket is about has not been resolved.

​https://github.com/evanw/esbuild/issues/81#issuecomment-680147925 says go can be built on 10.9 using legacy support and DYLD_INSERT_LIBRARIES, in case that's helpful.

[Ionic (Mihai Moldovan)]

That information won't help for building go, merely for using the pre-built binaries (unless something else doesn't fit).

Are you sure that the port doesn't build on 10.9+? Since the legacy-support has been updated to 0.12, go should build as well. When I pushed the change to the go port, legacy-support was still too old (and hence the automatic builds failed), but it did work with legacy-support-devel (for me and others).

I'm not sure if go has seen updates (version bumps, revbumps) since then. If it did, it's not unthinkable that the patching fails (again) and will have to be updated. I'd have to check.

[Wowfunhappy (Jonathan)]

I still can't build Go via MacPorts on 10.9.5, even after installing legacy-support-devel. However, I'm running into different missing symbols, specifically those listed at: #60611

So maybe the problem in this ticket has been fixed, but Go still won't build due to #60611?

[kencu (Ken)]

I think the full version of the current go is a rather long shot on older systems, but we will try our very best to use all our trickery here.

For interest, the big issue with go is the "C" bindings. I can get go version 1.11.5 building easily as far back as SnowLeopard <​https://github.com/kencu/SnowLeopardPorts/blob/master/lang/go/Portfile> with the C bindings disabled, and that actually lets a lot of ports that require go build, if that is of any use to you.

Getting go to support the C bindings on systems where upstream has dumped support is tricky, though. If you have skills in this area, much appreciate your help. Specifically, making go link in accessory libraries like legacy-support is a supreme PITA.

[Ionic (Mihai Moldovan)]

Can't we close this issue back up? The initial issue described here has been fixed, if I'm not mistaken.

The one in #60611 is caused by something else.

Or do we want to close #60221 as a duplicate of this one and keep this open?

[barracuda156]

Replying to Ionic:

Can't we close this issue back up? The initial issue described here has been fixed, if I'm not mistaken.

The one in #60611 is caused by something else.

Or do we want to close #60221 as a duplicate of this one and keep this open?

I am not sure it is fixed: ​https://trac.macports.org/ticket/69160#comment:8

[barracuda156]

Replying to Ionic:

Good, that seems to work:

%  nm -arch x86_64 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep -i fdopendir
0000000000002aa5 T _fdopendir
0000000000002bb1 T _fdopendir$INODE64
%  nm -arch i386 $(port work legacy-support)/destroot/opt/local/lib/libMacportsLegacySupport.dylib | grep -i fdopendir  
00002a55 T _fdopendir
00002c6f T _fdopendir$INODE64$UNIX2003
00002b62 T _fdopendir$UNIX2003

It is not the case anymore, legacy-support does not provide these now:

10:~ svacchanda$ port -v installed legacy-support | grep active
  legacy-support @1.1.1_0+universal (active) requested_variants='+universal' platform='darwin 10' archs='i386 x86_64' date='2024-01-26T03:23:00+0800'

10:~ svacchanda$ nm -arch i386 /opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir
000031ff T _fdopendir$INODE64$UNIX2003

10:~ svacchanda$ nm -arch x86_64 /opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir
00000000000041de T _fdopendir$INODE64

[kencu (Ken)]

Hmm. As you know, I have hung back my personal working 10.6 setup due to all the recent uproar in MacPorts on the older systems. I have a slightly older version of legacy-support-devel installed that is working really well for me (including supporting go). Here is what I have:

$ port -v installed legacy-support-devel | grep active
  legacy-support-devel @20221029_0+universal (active) requested_variants='+universal' platform='darwin 10' archs='i386 x86_64' date='2022-12-23T18:58:54-0800'

$ nm -arch i386 /opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir
00004607 T _fdopendir
00004899 T _fdopendir$INODE64$UNIX2003
00004750 T _fdopendir$UNIX2003

$ nm -arch x86_64 /opt/local/lib/libMacportsLegacySupport.dylib | grep fdopendir
0000000000004b6e T _fdopendir
0000000000004cb6 T _fdopendir$INODE64

[kencu (Ken)]

It would appear that somewhere along the line, something has been screwed up.

for interest, my current (working) version of go on 10.6 x86_64 is:

$ go version
go version go1.11.5 darwin/amd64