glib2: compilation issues with clang, for 10.5

[rmottola (Riccardo)]

On Leopard 10.5 64bit

Default compile is qith clang 3.7:

Undefined symbols for architecture x86_64:
  "___udivti3", referenced from:
      _g_get_monotonic_time in gmain.c.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[97/1192] /opt/local/bin/clang-mp-3.7 -Iglib/gtester.p -Iglib -I../glib-2.72.4/glib -I. -I../glib-2.72.4 -I/opt/local/include -fcolor-diagnostics -std=gnu99 -O2 -g -D_GNU_SOURCE -fno-strict-aliasing -DG_ENABLE_DEBUG -Wimplicit-fallthrough -Wunused -Wno-unused-parameter -Wno-pedantic -Wno-format-zero-length -Wno-variadic-macros -Werror=format=2 -Werror=init-self -Werror=missing-include-dirs -Werror=pointer-arith -Wstrict-prototypes -Wno-bad-function-cast -Wno-declaration-after-statement -Werror=implicit-function-declaration -Werror=missing-prototypes -pipe -Os -fstrict-aliasing -Wno-deprecated-declarations -arch x86_64 -UG_DISABLE_ASSERT -MD -MQ glib/gtester.p/gtester.c.o -MF glib/gtester.p/gtester.c.o.d -o glib/gtester.p/gtester.c.o -c ../glib-2.72.4/glib/gtester.c
ninja: build stopped: subcommand failed.
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_glib2/glib2/work/build" && /opt/local/bin/ninja -j2 --verbose -v 
Exit code: 1

if I force clang 6.0 (and 7.0) instead:

/opt/local/bin/clang++-mp-6.0 -Iglib/tests/cxx.p -Iglib/tests -I../glib-2.72.4/glib/tests -I. -I../glib-2.72.4 -Iglib -I../glib-2.72.4/glib -I/opt/local/include -fcolor-diagnostics -O2 -g -Wimplicit-fallthrough -Wunused -Wno-unused-parameter -Wno-pedantic -Wno-format-zero-length -Wno-variadic-macros -Werror=format=2 -Werror=init-self -Werror=missing-include-dirs -Werror=pointer-arith -pipe -Os -stdlib=libstdc++ -arch x86_64 -MD -MQ glib/tests/cxx.p/cxx.cpp.o -MF glib/tests/cxx.p/cxx.cpp.o.d -o glib/tests/cxx.p/cxx.cpp.o -c ../glib-2.72.4/glib/tests/cxx.cpp
In file included from ../glib-2.72.4/glib/tests/cxx.cpp:18:
In file included from ../glib-2.72.4/glib/glib.h:32:
In file included from ../glib-2.72.4/glib/gasyncqueue.h:32:
In file included from ../glib-2.72.4/glib/gthread.h:32:
In file included from ../glib-2.72.4/glib/gatomic.h:28:
../glib-2.72.4/glib/glib-typeof.h:39:10: fatal error: 'type_traits' file not found
#include <type_traits>
         ^~~~~~~~~~~~~
1 error generated.

I suppose this is a c++11 issue which is not correctly activated?

GCC 7 compiles!

[rmottola (Riccardo)]

I wonder if GCC "compiles" but fails to work? #67295

[jmroot (Joshua Root)]

Even clang-3.4 has support for C++14. __udivti3 is a function that should be provided by clang's runtime library compiler_rt.

[jmroot (Joshua Root)]

The type_traits issue was previously reported in #67268.

[kencu (Ken)]

did you fix your leopard sdk?

LeopardSDKFixes

once you do, your missing symbols should be found.

The type_traits issue has to do with the c++ standard not being set. gcc defaulted to c++11 as of about gcc5. clang defaulted to c++03 until about clang-10 or so, then defaults to c++17 after that.

So best to just set the c++ standard you want (need) to avoid these differences.

[rmottola (Riccardo)]

@kencu - sorry my fault. I checked, and I did not apply SDK fixes. I have two identical-.looking MacBooks, both white, both 10.5, but one is 32bit and one 64bit and the 64bit had still the original dylib.

Once the same dylib put in place (it is anyway universal binary x86-32, x86-64 and PPC-32), glib2 compiles with clang 3.7.

type_traits fails, but it means the two machines are on the same level.

Replying to kencu:

did you fix your leopard sdk?

LeopardSDKFixes

once you do, your missing symbols should be found.

[rmottola (Riccardo)]

I tried adding

configure.cxxflags-append -std=c++11

and compiling with clang 6.0, but I still get:

/opt/local/bin/clang++-mp-6.0 -Iglib/tests/cxx.p -Iglib/tests -I../glib-2.72.4/glib/tests -I. -I../glib-2.72.4 -Iglib -I../glib-2.72.4/glib -I/opt/local/include -fcolor-diagnostics -O2 -g -Wimplicit-fallthrough -Wunused -Wno-unused-parameter -Wno-pedantic -Wno-format-zero-length -Wno-variadic-macros -Werror=format=2 -Werror=init-self -Werror=missing-include-dirs -Werror=pointer-arith -pipe -Os -std=c++11 -stdlib=libstdc++ -arch x86_64 -MD -MQ glib/tests/cxx.p/cxx.cpp.o -MF glib/tests/cxx.p/cxx.cpp.o.d -o glib/tests/cxx.p/cxx.cpp.o -c ../glib-2.72.4/glib/tests/cxx.cpp
In file included from ../glib-2.72.4/glib/tests/cxx.cpp:18:
In file included from ../glib-2.72.4/glib/glib.h:32:
In file included from ../glib-2.72.4/glib/gasyncqueue.h:32:
In file included from ../glib-2.72.4/glib/gthread.h:32:
In file included from ../glib-2.72.4/glib/gatomic.h:28:
../glib-2.72.4/glib/glib-typeof.h:39:10: fatal error: 'type_traits' file not found
#include <type_traits>
         ^~~~~~~~~~~~~
1 error generated.

we can see in the command line that the option was added.

[kencu (Ken)]

this needs fixing too:

-stdlib=libstdc++

as that stdlib, the system default, does not support c++11.

Usually what is done is you add this to the Portfile:

compiler.cxx_standard 2011

and base adds the right things to make it work.

[mascguy (Christopher Nielsen)]

Riccardo, it might be better to install/fix glib2-upstream instead, as that will ultimately be the basis for glib2.

For your awareness, glib2 will be updated to the next major release - matching glib2-devel - within the next few weeks.

After that, we'll probably wait another 30-ish days, and finally update glib2 to the latest-and-greatest (matching glib2-upstream).

So to avoid having to repeat all of this again two more times, I'd suggest working with glib2-upstream.

Does that make sense?

[rmottola (Riccardo)]

As you wish, Christopher. I start trying to build glib2-upstream and report issues on ​https://trac.macports.org/ticket/67352

As I understand, things fixed there will come over to glib2 and we will close this one. Right now, I see more issues. I don't know if new or not. I'll try some clang and gcc combinations.

[mascguy (Christopher Nielsen)]

Replying to rmottola:

As you wish, Christopher. I start trying to build glib2-upstream and report issues on ​https://trac.macports.org/ticket/67352

As I understand, things fixed there will come over to glib2 and we will close this one. Right now, I see more issues. I don't know if new or not. I'll try some clang and gcc combinations.

I definitely don't want to cause you undue pain, or have you start over. But at the same time, more updates are coming in the near future, too.

Ken, what do you think? Would it be better for Riccardo to continue with fixing glib2, or focus on glib2-upstream...?

[rmottola (Riccardo)]

Ok, let's continue here. This comment is for glib2-upstream on 32bit intel.

Might be, most people stick to 32bit even if he CPU is 64bit capable or upgraded to 10.6. I have two twin machines. Does this mean that the 64bit libs are "poor" ? where should these symbols be (which library?)

I tried on the other 32bit and I get to the standard error:

In file included from ../glib-2.76.2/glib/tests/cxx.cpp:20:
In file included from ../glib-2.76.2/glib/glib.h:34:
In file included from ../glib-2.76.2/glib/gasyncqueue.h:34:
In file included from ../glib-2.76.2/glib/gthread.h:34:
In file included from ../glib-2.76.2/glib/gatomic.h:30:
../glib-2.76.2/glib/glib-typeof.h:43:10: fatal error: 'type_traits' file not found
#include <type_traits>
         ^~~~~~~~~~~~~
1 error generated.

I added

configure.cxxflags-append -std=c++11

but it still doesn't work.

[rmottola (Riccardo)]

I report here also 64bit glib2-upstream, where on 64bit 10.5 it fails

Undefined symbols for architecture x86_64:
  "_close$NOCANCEL$UNIX2003", referenced from:
      _g_file_get_contents in gfileutils.c.o
      _write_to_file in gfileutils.c.o
      _g_child_watch_finalize in gmain.c.o
      _g_close in gstdio.c.o
      _g_test_trap_fork in gtestutils.c.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[kencu (Ken)]

the symbols are in libSystem.dylib on 10.5:

$ nm -a -arch i386 libSystem.B.dylib | grep _close
0008e3ff T ___bt_close
000e0e14 T ___rec_close
00113604 T ___res_close
0010fb0f T __res_close
00041748 T _asl_close
000eeb70 T _asl_file_close
000eefc2 T _asl_file_list_close
000f1bcf T _asl_legacy1_close
000edd13 T _asl_store_close
000edef6 T _asl_store_file_close
000edcaf T _asl_store_file_closeall
00001ca0 T _close
00001ca0 T _close$NOCANCEL$UNIX2003
0000c848 T _close$UNIX2003
00085dd1 T _closedir
00024e6b T _closedir$UNIX2003
00067150 T _closelog
000a7a73 T _dbm_close
0002f42d T _fts_close
00084784 T _fts_close$INODE64
000911f4 T _kvm_close
001292dc T _launchd_close
0008e4ff T _mpool_close
0007dfbc T _sem_close

$ nm -a -arch x86_64 libSystem.B.dylib | grep _close
0000000000084966 T ___bt_close
00000000000ce0c0 T ___rec_close
00000000000f8cce T ___res_close
00000000000f57c6 T __res_close
000000000003e33d T _asl_close
00000000000da6e2 T _asl_file_close
00000000000daadc T _asl_file_list_close
00000000000dceca T _asl_legacy1_close
00000000000d95cd T _asl_store_close
00000000000d97bd T _asl_store_file_close
00000000000d9568 T _asl_store_file_closeall
0000000000021da8 T _close
00000000000e75c8 T _close$NOCANCEL
000000000007c9d5 T _closedir
000000000006152b T _closelog
0000000000099748 T _dbm_close
000000000002c438 T _fts_close
000000000007b24a T _fts_close$INODE64
000000000010d630 T _launchd_close
0000000000084a71 T _mpool_close
0000000000075158 T _sem_close

as you can see, this one only exists for 32 bit:

_close$NOCANCEL$UNIX2003

[kencu (Ken)]

So for all the past many years, when hacks are done for older systems in various software, proposed by helpful folks who often don't know the deep details of stuff like this, they only support the "common pathways" like building 32 bit on 10.5.

It's an error somewhere -- some header, some build item building for the wrong arch, some hack in the source of something -- could be our error, in something like legacysupport, or glib2's error in some hack.

But because building 64 bit on 10.5 is a very uncommon pathway, nobody has stumbled across it before, or if they did, they haven't pushed the fix through to where it needs to go.

Step one is you have to find it -- somewhere where the 64 bit pathway is wrong. Then fixing it is probably not that hard. It takes time.

What you need to decide is whether building 64 bit software on 10.5 is worth the extra hassle. Maybe it is. Some things fail to build as 32bit now, because nothing is tested against 32 bit any more in the macOS world. Nothing.

[kencu (Ken)]

Replying to rmottola:

I added

configure.cxxflags-append -std=c++11

but it still doesn't work.

With gcc5+ as the compiler, it should work. gcc5+ defaults to c++11 as the standard, and automatically uses it's newer libstdc++.dylib and newer headers.

With clang as the compiler, you will have to force non-defaults. You would have to add the -std=c++11 to the cxxflags as you did, but then you also need to force clang to use a newer standard library.

The usual one on macOS is libc++, so you would make this happen on the cxxflags and ldflags -stdlib=libc++ for most MacPorts builds. This is what we default to and test on 10.6+.

For our "bastardized" builds on 10.4 and 10.5, we have the hack that lets clang use libgcc's headers and libraries. So you have to make this happen with the cxxflags (and ldflags) -stdlib=macports-libstdc++. Usually that is done by base, by setting this in the Portfile:

compiler.cxx_standard 2011

but then you will have to inspect the build lines to make sure that you are not setting the stdlib twice (that may not work out if they are different) and that the stdlib setting is also showing up in the ldflags (otherwise the link will fail).

[Cebtenzzre]

I was able to get glib2 @2.74.7_0+x11 and glib2-devel @2.76.2_0+x11 to build on OS X Leopard 10.5.8 i386 with Xcode 3.1.4 by simply adding compiler.cxx_standard 2011 to the Portfile.

[Christopher Nielsen <mascguy@…>]

In 988da2794e36195e37d62910a01f4f9351df4796/macports-ports (master):

glib2*: set cxx_standard to 2011
See: #67307

[evanmiller (Evan Miller)]

The missing _close$NOCANCEL$UNIX2003 symbol is declared explicitly in glib/gstdioprivate.h in order to access a hidden Darwin API which apparently is not present on 64-bit Leopard (or on Tiger). Commenting out the #define close close$NOCANCEL$UNIX2003 in that file should be sufficient to get things to compile, with the minor side-effect that calls to close will be cancellable from other threads.

[kencu (Ken)]

yes, there it is, indeed:

#ifdef __APPLE__
#include <sys/cdefs.h>
#include <unistd.h>
# if !__DARWIN_NON_CANCELABLE
#  if !__DARWIN_ONLY_UNIX_CONFORMANCE
#   define close close$NOCANCEL$UNIX2003
int close$NOCANCEL$UNIX2003 (int fd);
#  else
#   define close close$NOCANCEL
int close$NOCANCEL (int fd);
#  endif
# endif
#endif

we'd need for force this one to be used on 64 bit Leopard, which apparently presently is not being used:

close$NOCANCEL

I don't know about Tiger just now, 32 or 64 bit. I suspect both use this: close$NOCANCEL$UNIX2003 but have to check.

I don't know what this is supposed to do: __DARWIN_ONLY_UNIX_CONFORMANCE

[evanmiller (Evan Miller)]

Opened a PR with a Tiger fix: ​https://github.com/macports/macports-ports/pull/18677

Feel free to suggest a change to that addressing 64-bit Leopard.

[kencu (Ken)]

long description of the (ancient) issues here:

​https://chromium.googlesource.com/chromium/src/base/+/refs/heads/main/mac/close_nocancel.cc

but that has no logic to support a Leopard 64-bit implementation in it, as we can see (and is why Riccardo sees issues with that build).

[kencu (Ken)]

For completeness here, Tiger has only the basic close calls:

TigerVM:~$ nm -a -arch x86_64 /usr/lib/libSystem.B.dylib | grep _close | grep T
00000000000bb24e T ___bt_close
00000000000b1744 T ___rec_close
00000000000e4440 T ___res_close
00000000000e0dfb T __res_close
0000000000035ffa T _asl_close
00000000000d17ec T _close
000000000009f165 T _closedir
000000000007c678 T _closelog
000000000003e989 T _dbm_close
000000000002ff5c T _fts_close
00000000000d1bfd T _mpool_close
000000000004167c T _sem_close

TigerVM:~$ nm -a -arch i386 /usr/lib/libSystem.B.dylib | grep _close | grep T
900874d6 T _closedir$UNIX2003
9004735b T _fts_close
900b204d T _asl_close
900b8f2a T _dbm_close
9006d490 T _sem_close
9004509c T _closelog
90016440 T _closedir
900f8a46 T ___rec_close
9004cb4f T ___bt_close
9000fdf0 T _close
9004cd30 T _mpool_close
90115aec T __res_close
901190d0 T ___res_close
90067b3d T _kvm_close
901586f3 T _launchd_close

so none of the $NOCANCEL or $UNIX2003 _close variants appear to be available there. (_closedir$UNIX2003 for i386 only noted).

[evanmiller (Evan Miller)]

In 77685f6bf9ff430984905af0c7ee0b2b83fd944b/macports-ports (master):

glib2: fix build on Tiger, remove legacysupport

Closes: #67444
Closes: #67451
See: #67307

[mascguy (Christopher Nielsen)]

I finally bit the bullet, and installed clang-7.0 on my 2006-era MacBookPro running 10.5-i386. The biggest hurdle was waiting for gcc10-bootstrap to build, which literally took several days. I didn't time it - should have kept the build log, to get an accurate figure - but wouldn't be surprised if it took 48 hours. (The time was split over multiple days of 8-hour sessions; don't like leaving an old machine like this cranking away when I'm absent or asleep.)

Anyhow, that's on a dual-core Intel Core CPU, running at 2.16 GHz. So I can't even fathom trying to build this stack on my 2002 PowerBook G4, as I expect it to take up to 4 times longer. That's a huge barrier to entry, so if we can ever get some type of 10.5_x32 builder back online - and publish public toolchain binaries - that would be tremendously helpful! (Currently there's active discussion surrounding this, via issue:66288.)

Anyhow, with the requisite toolchain in place, I was able to successfully build glib2... either Quartz or X11. (Albeit 32-bit, and non-universal.)

Riccardo, are you still having issues building on 10.5-x64?

[mascguy (Christopher Nielsen)]

Has duplicate issue:67483

[ballapete (Peter "Pete" Dyballa)]

glib2-devel and glib2 @2.76.2_2 have a little difference in Portfile:

195,197d193
<     # https://trac.macports.org/ticket/67307
<     configure.cflags-append -D__DARWIN_NON_CANCELABLE=1
< 

glib2-devel @2.76.2_2, X11 variant, built for me on PPC Tiger, Mac OS X 10.4.11, Darwin 8 with GCC7. I put these three lines difference into the other Portfile and am building glib2 @2.76.2_2. Will take one more hour time.

[ballapete (Peter "Pete" Dyballa)]

glib2 @2.76.2_2 built with the additional lines from glib2-devel @2.76.2_2 Portfile.

[Christopher Nielsen <mascguy@…>]

In d6328dbc2ce02db9b3e825d3d8ff2394fc7a0b96/macports-ports (master):

glib2: tiger: add darwin_non_cancelable fix from glib2-devel
See: #67307

[rmottola (Riccardo)]

I tried rebuilding today on Leopard 64bit glib (clean build) I see the last fix is in the portfile (configure.cflags-append -DDARWIN_NON_CANCELABLE=)

glib2-devel fails the same. Compilation with gcc7.

Yet i still get the original problem I had in comment 12:

Undefined symbols for architecture x86_64:
  "_close$NOCANCEL$UNIX2003", referenced from:
      _write_to_file in gfileutils.c.o
      _g_file_get_contents in gfileutils.c.o
      _g_child_watch_finalize in gmain.c.o
      _g_close in gstdio.c.o
      _g_test_trap_fork in gtestutils.c.o
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status

[kencu (Ken)]

as I recall, that symbol close$NOCANCEL$UNIX2003 doesn't exist for a 64 bit build.

[rmottola (Riccardo)]

Sorry.... checked the

configure.cflags-append -DDARWIN_NON_CANCELABLE=

is only for darwin 8, I added it for darwin 9, then it compiles!

Indeed:

$ nm -a -arch i386 /usr/lib/libSystem.B.dylib | grep _close | grep T
0008e30f T ___bt_close
000e0e1c T ___rec_close
0011360c T ___res_close
0010fb17 T __res_close
00041658 T _asl_close
000eeb78 T _asl_file_close
000eefca T _asl_file_list_close
000f1bd7 T _asl_legacy1_close
000edd1b T _asl_store_close
000edefe T _asl_store_file_close
000edcb7 T _asl_store_file_closeall
00001b80 T _close
00001b80 T _close$NOCANCEL$UNIX2003
0000c728 T _close$UNIX2003
00085ce1 T _closedir
00024d4b T _closedir$UNIX2003
00067060 T _closelog
000a7983 T _dbm_close
0002f30d T _fts_close
00084694 T _fts_close$INODE64
00091104 T _kvm_close
001292e4 T _launchd_close
0008e40f T _mpool_close
0007decc T _sem_close

but:

$ nm -a -arch x86_64 /usr/lib/libSystem.B.dylib | grep _close | grep T
00000000000847d6 T ___bt_close
00000000000ce0dc T ___rec_close
00000000000f8cea T ___res_close
00000000000f57e2 T __res_close
000000000003e1cd T _asl_close
00000000000da6fe T _asl_file_close
00000000000daaf8 T _asl_file_list_close
00000000000dcee6 T _asl_legacy1_close
00000000000d95e9 T _asl_store_close
00000000000d97d9 T _asl_store_file_close
00000000000d9584 T _asl_store_file_closeall
0000000000021c08 T _close
00000000000e75e4 T _close$NOCANCEL
000000000007c865 T _closedir
00000000000613bb T _closelog
00000000000995b8 T _dbm_close
000000000002c298 T _fts_close
000000000007b0da T _fts_close$INODE64
000000000010d64c T _launchd_close
00000000000848e1 T _mpool_close
0000000000074fe8 T _sem_close

so the hack is needed on darwin 9 intel-64bit only, to be precise.

[rmottola (Riccardo)]

This fix still applies. I need to move out

configure.cflags-append -DDARWIN_NON_CANCELABLE=

from the darwin 8 check.

I would say this is now a duplicate of #68425