Coreutils fails to build with XCode 11 on 10.15

[markemer (Mark Anderson)]

Trying to compile coreutils and I get the following compiler error.

depbase=`echo lib/ftello.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
	/usr/bin/clang  -I. -I./lib  -Ilib -I./lib -Isrc -I./src -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk   -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -MT lib/ftello.o -MD -MP -MF $depbase.Tpo -c -o lib/ftello.o lib/ftello.c &&\
	mv -f $depbase.Tpo $depbase.Po
lib/ftello.c:53:12: error: no member named '_flag' in 'struct __sFILE'; did you mean '_flags'?
  if (fp_->_flag & _IOWRT)
           ^~~~~
           _flags
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:130:8: note: '_flags' declared here
        short   _flags;         /* flags, below; this FILE is free if 0 */
                ^
lib/ftello.c:53:20: error: use of undeclared identifier '_IOWRT'
  if (fp_->_flag & _IOWRT)
                   ^
lib/ftello.c:64:21: error: no member named '_flag' in 'struct __sFILE'; did you mean '_flags'?
          if ((fp_->_flag & _IONBF) == 0 && fp_->_base != NULL)
                    ^~~~~
                    _flags
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:130:8: note: '_flags' declared here
        short   _flags;         /* flags, below; this FILE is free if 0 */
                ^
lib/ftello.c:64:50: error: no member named '_base' in 'struct __sFILE'
          if ((fp_->_flag & _IONBF) == 0 && fp_->_base != NULL)
                                            ~~~  ^
lib/ftello.c:65:25: error: no member named '_ptr' in 'struct __sFILE'
            pos += fp_->_ptr - fp_->_base;
                   ~~~  ^
lib/ftello.c:65:37: error: no member named '_base' in 'struct __sFILE'
            pos += fp_->_ptr - fp_->_base;
                               ~~~  ^
6 errors generated.
make[2]: *** [lib/ftello.o] Error 1
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_src_macports_macports-ports_sysutils_coreutils/coreutils/work/coreutils-8.31'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_src_macports_macports-ports_sysutils_coreutils/coreutils/work/coreutils-8.31'
make: *** [all] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_src_macports_macports-ports_sysutils_coreutils/coreutils/work/coreutils-8.31'
Command failed:  cd "/opt/local/var/macports/build/_opt_src_macports_macports-ports_sysutils_coreutils/coreutils/work/coreutils-8.31" && /usr/bin/make -w all 
Exit code: 2

It's related to this in autoconf I think:

checking whether ftello is declared... yes
checking for ftello... yes
checking whether ftello works... no

Because as far as I know ftello() works fine on the Mac. So the question becomes, is this upstream Apple, upstream GNU or actually us?

[markemer (Mark Anderson)]

Log file of coreutils build with port -ucpvd

[yan12125 (Chih-Hsuan Yen)]

So the question becomes, is this upstream Apple, upstream GNU or actually us?

Dunno. I can't reproduce the issue with Xcode 11.0 on 10.14.6, and for various I can't upgrade to 10.15 in near future, even after it's official release in October.

So, I will debug it blindly. First could you also attach /opt/local/var/macports/build/_opt_src_macports_macports-ports_sysutils_coreutils/coreutils/work/coreutils-8.31/config.log?

[kencu (Ken)]

These guys <​https://gitmemory.com/issue/fink/fink/197/531629669> say to unset this interesting flag:

FTELLO_BROKEN_AFTER_SWITCHING_FROM_READ_TO_WRITE

and Google dishes up 254 similar instances in 0.48s :>

[kencu (Ken)]

Here's an explanation from a homebrew discussion <​https://discourse.brew.sh/t/mac-os-10-15-catalina-issues/5001/22>:

An update. Guess I figured out the cause, but still no idea how to fix it.

The configure script for the gnulib component, which replaces “broken” libc functions with its own versions, checks for some undesired behavior of ftell(). It detects that on macOS, if you open a file in “r+” mode, read it up to EOF (by fseek()ing to its end, really), write one more byte, and call ftell() on it, the reported position will be wrong — the file pointer needs to advance as you write more data. There seems to be a known bug on Solaris, just like that.

So the configure script sets a variable telling that ftell() is broken on this system, but gnulib only has a replacement for Solaris, which uses the struct __sFILE as found there, and predictably barfs because __sFILE on Darwin is nothing like that.

How healthy an approach to patching systems that misbehave is left as an exercise to the reader, but here’s one thing: up to and including Mojave, ftell() was working just fine, and the replacement was never needed. Maybe, hm, someone more in the know should report it to Apple before they make a release?

raduc I was able to build gettext by setting FTELLO_BROKEN_AFTER_SWITCHING_FROM_READ_TO_WRITE to 0 as described here: ​https://gitmemory.com/issue/fink/fink/197/531629669

[markemer (Mark Anderson)]

Hey, thanks for the quick replies. I'm going to give that FTELLO_BROKEN_AFTER_SWITCHING_FROM_READ_TO_WRITE - I'm certain that is the deal - I'll report back. Although, it makes me think I should file a radar about ftell() misbehaving.

[markemer (Mark Anderson)]

Ken, that was 100% what it was. I'm going to go digging to see what the hell is ACTUALLY going on, but everything seems to work.

[yan12125 (Chih-Hsuan Yen)]

Thank you Ken for finding those materials! From them, I'm now relatively confident that the issue is from GNU and/or Apple, and there is not much MacPorts can do. Like Yaroslav Fedevych said in ​https://discourse.brew.sh/t/mac-os-10-15-catalina-issues/5001/32:

You can of course set that variable, doesn’t mean your program won’t corrupt your data in the runtime because of this ftell() behavior.

Given that Apple is relatively slow in fixing their systems, I think this issue should be forwarded to GNU. I attached the test program from gnulib as ftello-test.c. Report the outcome of that program and the content of conftest.tmp to the bug-gnulib mailing list (1) should help GNU gurus write another workaround for Catalina's issue.

(1) ​https://lists.gnu.org/mailman/listinfo/bug-gnulib

[yan12125 (Chih-Hsuan Yen)]

From ​https://trac.macports.org/ticket/58916#comment:11,

ftell() bug is fixed on macOS 10.15 Beta 10

So coreutils should be fine now. Feel free to reopen or ping me if it's still broken.