go: Undefined symbols _macports_legacy_sysconf

[Wowfunhappy (Jonathan)]

Error: Failed to build go: command execution failed

A full copy of the log is attached. However, the relevant section appears to be:

:info:build Undefined symbols for architecture x86_64:
:info:build   "_macports_legacy_sysconf", referenced from:
:info:build       __cgo_a722341dd114_Cfunc_sysconf in _x002.o
:info:build ld: symbol(s) not found for architecture x86_64

Please let me know if you need anything else!

[ryandesign (Ryan Carsten Schmidt)]

This particular error is new to me and the occurrence of "macports_legacy" in the undefined symbol name may mean this is a problem with MacPorts legacy support that we can resolve.

However, there have been many other reports of build failures of go on older systems, so the most reliable and future-proof way to get go on macOS is to upgrade to a newer version of macOS.

[kencu (Ken)]

I was trying to figure out where Chris was going with this uncommitted branch <​https://github.com/macports/macports-legacy-support/commit/8f5629a76f9435a2da0d2d7948bfa73107e23376> and perhaps it was for something like this...

[kencu (Ken)]

I am not certain that @ionic ever fully finished his legacysupport tweaks that were designed exactly to get go working on older systems. We did a lot of work in legacysupport for this (some of which I understand :> ) and maybe there is still a bit to do.

[kencu (Ken)]

FWIW -- the bad part of go on older sytems is the C bindings. I actually have a fairly recent go running on 10.6.8 with the C bindings disabled.

<​https://github.com/kencu/SnowLeopardPorts/blob/master/lang/go/Portfile>

a lot of ports that need go to build seem to be OK without the C bindings.

[Wowfunhappy (Jonathan)]

I was trying to see if I could resolve the issue by using an older version of Go.

Go 1.12.1 was explicitly tested and confirmed to work on 10.7. ​https://github.com/macports/macports-ports/commit/f15f259d315c132cce4fb607cfac54ceaceef99d However, for me on 10.9, it fails with the same error as the current version.

I don't know very much about the internals of MacPorts, but this would seem to leave two possibilities: (A) Go 1.12.1 worked on 10.7, but not 10.9, or (B) this is actually a regression in the legacy-support package, which occurred between 0.10_0 (current version at the time of above commit) and 0.12_0 (currently current version).

(I could not figure out how to tell MacPorts to install Go without upgrading legacy-support.)

[kencu (Ken)]

The go port had a bunch of surgery done to it by @ionic to make the legacysupport PG work <​https://github.com/macports/macports-ports/commit/abe46d0764f08fdec9a4923880ce03edbc768a0d>

and then he stripped a lot of it out <​https://github.com/macports/macports-ports/commit/bcfdbd30124a6a6cbfa3fd1afc2da704f6d32b92#diff-5cddd3daff0d75a83514cc9e2aa95e8b> when he said it was incorporated upstream.

That might be why 10.9 is having troubles.

We have a bail-out define to turn off function wrapping for just such eventualities, while this gets sorted out:

<​https://github.com/macports/macports-legacy-support/blob/170f17c6b08ea67532f25e399749ff29e1fbc678/include/MacportsLegacySupport.h#L100>

I'm not that good at go but if you can add this somehow #define __DISABLE_MP_LEGACY_SUPPORT_SYSCONF_WRAP__ you will turn that part of legacysupport off, while Ionic sorts it out.

[ryandesign (Ryan Carsten Schmidt)]

Replying to Wowfunhappy:

(I could not figure out how to tell MacPorts to install Go without upgrading legacy-support.)

It should not normally be used, but for testing purposes, you can accomplish this using port's -n switch.

[kencu (Ken)]

Or you could just do a quick edit in the correct legacysupport header to define that

#define __DISABLE_MP_LEGACY_SUPPORT_SYSCONF_WRAP__

build go, and then switch it back if you get really frustrated :>

[erikbs]

Thanks, adding

#define __DISABLE_MP_LEGACY_SUPPORT_SYSCONF_WRAP__ 1

(note the value, 1) immediately after

#define _MACPORTS_LEGACYSUPPORTDEFS_H_

in MacportsLegacySupport.h allowed me to compile go successfully on 10.9.5.

I did, however, encounter some problems with the recommended environment variables, i.e.:

GO_EXTLINK_ENABLED="1"
GO_LDFLAGS="\"-extldflags=-L/opt/local/lib -Wl,-headerpad_max_install_names -lMacportsLegacySupport\""

For some reason go run does not seem to honor them and just spits out the following error:

# command-line-arguments
/opt/local/lib/go/pkg/tool/darwin_amd64/link: running /usr/bin/clang failed: exit status 1
Undefined symbols for architecture x86_64:
  "_fdopendir$INODE64", referenced from:
      _syscall.libc_fdopendir_trampoline in go.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Passing the values through the -ldflags parameter instead does seem to work, at least with a ​simple test program:

go run -ldflags "-linkmode external -extldflags \"-L/opt/local/lib -Wl,-headerpad_max_install_names -lMacportsLegacySupport\"" hello.go

---

Edit: fdopendir does not seem to function properly. When calling go get -u go.chromium.org/luci/cipd/client/cmd/cipd, with or without the -ldflags parameter, I get lots of fdopendir: too many open files errors.

[Wowfunhappy (Jonathan)]

See also: #60221

[ryandesign (Ryan Carsten Schmidt)]

Replying to kencu:

a lot of ports that need go to build seem to be OK without the C bindings.

Could the Go C bindings be separated out into a subport?

[Ionic (Mihai Moldovan)]

I don't get this issue.

Not like in "cannot reproduce it", which I can, but I don't understand why it happens.

More curiously, I was able to install go 1.14 successfully with a development version of legacy-support (including the sysconfig wrapper and my changes) back then, about a year ago. I'm getting the same error now, even after downgrading the legacy-support and go ports to exactly the same versions.

Replying to kencu:

The go port had a bunch of surgery done to it by @ionic to make the legacysupport PG work <​https://github.com/macports/macports-ports/commit/abe46d0764f08fdec9a4923880ce03edbc768a0d>

and then he stripped a lot of it out <​https://github.com/macports/macports-ports/commit/bcfdbd30124a6a6cbfa3fd1afc2da704f6d32b92#diff-5cddd3daff0d75a83514cc9e2aa95e8b> when he said it was incorporated upstream.

That might be why 10.9 is having troubles.

Everything that I backed out was either redundant, since incorporated upstream, or obsolete, since recent base versions support setting environment variables in a much saner way.

I'm a bit stumped by this and debugged this a whole day already, without being able to even successfully rebuild the initial configuration that was "known good" to me. I haven't tried downgrading base yet, but will probably have to do so, even if I cannot imagine how any changes there could cause the issue we're seeing here. Really odd...

[Ionic (Mihai Moldovan)]

I finally was able to enable some debugging to my go build (after locally debugging and fixing errors when using the "-x" go flag for days...) and here's finally the gist:

:info:build os/user
:info:build mkdir -p $WORK/b035/
:info:build cd /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_lang_go/go/work/go/src/os/user
:info:build CGO_LDFLAGS='"-g" "-O2"' /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_lang_go/go/work/go/pkg/tool/darwin_amd64/cgo -objdir $WORK/b035/ -importpath os/user -- -I $WORK/b035/ -g -O2 ./cgo_lookup_unix.go ./getgrouplist_darwin.go ./listgroups_unix.go
:info:build cd $WORK/b035
:info:build TERM='dumb' /usr/bin/clang -I /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_lang_go/go/work/go/src/os/user -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -gno-record-gcc-switches -fno-common -I ./ -g -O2 -o ./_x001.o -c _cgo_export.c
:info:build TERM='dumb' /usr/bin/clang -I /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_lang_go/go/work/go/src/os/user -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -gno-record-gcc-switches -fno-common -I ./ -g -O2 -o ./_x002.o -c cgo_lookup_unix.cgo2.c
:info:build TERM='dumb' /usr/bin/clang -I /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_lang_go/go/work/go/src/os/user -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -gno-record-gcc-switches -fno-common -I ./ -g -O2 -o ./_x003.o -c getgrouplist_darwin.cgo2.c
:info:build TERM='dumb' /usr/bin/clang -I /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_lang_go/go/work/go/src/os/user -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -gno-record-gcc-switches -fno-common -I ./ -g -O2 -o ./_x004.o -c listgroups_unix.cgo2.c
:info:build TERM='dumb' /usr/bin/clang -I /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_lang_go/go/work/go/src/os/user -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -gno-record-gcc-switches -fno-common -I ./ -g -O2 -o ./_cgo_main.o -c _cgo_main.c
:info:build cd /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_lang_go/go/work/go/src/os/user
:info:build TERM='dumb' /usr/bin/clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -gno-record-gcc-switches -fno-common -o $WORK/b035/_cgo_.o $WORK/b035/_cgo_main.o $WORK/b035/_x001.o $WORK/b035/_x002.o $WORK/b035/_x003.o $WORK/b035/_x004.o -g -O2
:info:build # os/user
:info:build Undefined symbols for architecture x86_64:
:info:build   "_macports_legacy_sysconf", referenced from:
:info:build       __cgo_a722341dd114_Cfunc_sysconf in _x002.o
:info:build ld: symbol(s) not found for architecture x86_64
:info:build clang: error: linker command failed with exit code 1 (use -v to see invocation)

As already suspected, I can now say for sure that external linking is not done for this package, leading to an error. Also, I can finally see what exactly is being executed, so that's helpful. Maybe.

It seems that all cgo-based packages are not using the "usual" external linker LDFLAGS we pass it.

The fix, luckily, seems to be easy.

I'll push it once I confirm that it works with the newest version.

[Mihai Moldovan <ionic@…>]

In 0cd8874b6a94f3cd11c0b9d5d09532faa554afc4/macports-ports (master):

lang/go: use legacy-support LDFLAGS with cgo as well.

Our special LDFLAGS (when needed on older systems) are not being used
when building cgo-based packages, leading to link errors.

Pass them down via CGO_LDFLAGS (augmenting the default flags -g and -O2,
just to play it safe) and add this to the notes.

Fixes: #60611

[Ionic (Mihai Moldovan)]

Ken and others: can you confirm this to work?

It builds on 10.9 for me. Sadly, I cannot test any other platform.

Replying to erikbs:

Edit: fdopendir does not seem to function properly. When calling go get -u go.chromium.org/luci/cipd/client/cmd/cipd, with or without the -ldflags parameter, I get lots of fdopendir: too many open files errors.

I guess that should be a separate ticket. It sounds like legacy-support's implementation might be wrong, but I really have no idea what is causing that and, frankly, I, personally, don't want to put effort into finding out why it's misbehaving...

[Ionic (Mihai Moldovan)]

And of course, forgot to add myself to CC. Done.