gcc13 optimizer bug in ncurses

[ThomasDickey (Thomas Dickey)]

I noticed a problem in ncurses not displaying multibyte characters (used in line-drawing), and narrowed it down to a problem optimizing an expression in tty_update.c, using this chunk(​https://github.com/ThomasDickey/ncurses-snapshots/blob/a98f459acb17efd5c1754d12efdfeba8cdb70e68/ncurses/curses.priv.h#L1458):

				} else {					    \
				    int PUTC_j;					    \
				    for (PUTC_j = 0; PUTC_j < PUTC_n; ++PUTC_j) {   \
					TR_PUTC(PUTC_buf[PUTC_j]);		    \
					NCURSES_OUTC_FUNC (NCURSES_SP_ARGx PUTC_buf[PUTC_j]); \
				    }						    \
				}						    \

used here(​https://github.com/ThomasDickey/ncurses-snapshots/blob/a98f459acb17efd5c1754d12efdfeba8cdb70e68/ncurses/tty/tty_update.c#L357):

    PUTC(CHDEREF(ch));

There's no issue reported in valgrind, address sanitizer, etc., in other environments, which would indicate a problem in ncurses, but with optimization that goes through the loop the expected number of times but fails to call the output NCURSES_OUTC_FUNC more than once.

If it helps, I can attach the generated assembly for gcc12 (working) and gcc13 (nonworking).

[kencu (Ken)]

Not likely anyone on this board will be able to help you or sort this out.

If it is a general problem with gcc13 that you see on all systems, then it would best be reported as a gcc bug in the gcc bug tracker.

If it is a specific problem only with gcc13 on Darwin, on all arches, then it would also probably best be reported in the gcc bug tracker.

If it is a specific problem only with gcc13 on Darwin arm64, then you should report it here, to the person who is working on the Darwin arm64 implementation for gcc:

​https://github.com/iains/gcc-darwin-arm64/issues?q=is%3Aissue+is%3Aopen

[ThomasDickey (Thomas Dickey)]

I see this problem on amd64 and arm64 (each of my newer Macs), but not on other platforms where I have gcc 13 (Fedora 38 and 39, Slackware current), nor on one with gcc 14 (Fedora rawhide -- not a stable release). I've gcc 13 in a couple of other places, untested since gcc 12 is what's used.

However, since this affects the ncurses port, I started by reporting it here (in case there's some way to mark the port as conflicting with gcc13).

[kencu (Ken)]

macports can do that, yes.

It can mark gcc13 as an unusable compiler for that port.

[jmroot (Joshua Root)]

I have no particular insight into the specifics here, but I would recommend using clang's undefined behaviour sanitizer (-fsanitize=undefined) if you haven't already, just to rule that out.

[ryandesign (Ryan Carsten Schmidt)]

Replying to ThomasDickey:

(in case there's some way to mark the port as conflicting with gcc13).

Replying to kencu:

macports can do that, yes.

It can mark gcc13 as an unusable compiler for that port.

Well a port can indicate, using compiler.blacklist, that MacPorts should not choose some compiler(s) to build it. However, I don't see why MacPorts would ever have chosen gcc13 for ncurses to begin with. Since the port doesn't contain any directives to the contrary, MacPorts should have chosen clang.

Was ncurses built with gcc13 because you explicitly asked to build it with gcc13 (e.g. sudo port -s install ncurses configure.compiler=macports-gcc-13, which is not recommended)? If so, then MacPorts doesn't currently have a way to prevent that. Command-line compiler selections override compiler.blacklist.

[ThomasDickey (Thomas Dickey)]

I ran into this in developing ncurses (not the port) because I build with all of the compilers that are available (as well as testing with c89, c99 and c11/c23 however they're named).

But it hadn't occurred to me to see what the port uses.

[ThomasDickey (Thomas Dickey)]

I found the problem; it is simple to explain. ncurses uses the symbol MB_LEN_MAX, which has a fallback definition in limits.h -- which is too short to use -- while the "real" definition is lacking in gcc13 (since I've encountered the problem on three machines, that rules out "environment stuff"). The effect of the too-short definition is a buffer overrun. I'll make a workaround in ncurses, but it would be nice to fix the port.

(I'll see if I can find where the proper definition comes from in gcc12).

[ThomasDickey (Thomas Dickey)]

Using gcc12 to report a redefinition tells me (on my Ventura machine)

../ncurses/./tty/tty_update.c:218: warning: "MB_LEN_MAX" redefined
  218 | #define MB_LEN_MAX 20
      | 
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/usr/include/machine/limits.h:9,
                 from /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/usr/include/limits.h:64,
                 from /opt/local/lib/gcc12/gcc/x86_64-apple-darwin22/12.3.0/include-fixed/limits.h:203,
                 from /opt/local/lib/gcc12/gcc/x86_64-apple-darwin22/12.3.0/include-fixed/syslimits.h:7,
                 from /opt/local/lib/gcc12/gcc/x86_64-apple-darwin22/12.3.0/include-fixed/limits.h:34,
                 from ../ncurses/curses.priv.h:76,
                 from ../ncurses/./tty/tty_update.c:49:
/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/usr/include/i386/limits.h:45: note: this is the location of the previous definition
   45 | #define MB_LEN_MAX      6               /* Allow 31 bit UTF2 */
      | 

but gcc13 doesn't see those files:

../ncurses/./tty/tty_update.c:218: warning: "MB_LEN_MAX" redefined
  218 | #define MB_LEN_MAX 20
      | 
In file included from ../ncurses/curses.priv.h:76,
                 from ../ncurses/./tty/tty_update.c:49:
/opt/local/lib/gcc13/gcc/x86_64-apple-darwin22/13.2.0/include/limits.h:33: note: this is the location of the previous definition
   33 | #define MB_LEN_MAX 1
      |

Since my Sonoma machine gives the same misbehavior in ncurses, I think it's the same issue.

[ThomasDickey (Thomas Dickey)]

Looking at the include- and include-fixed directories in the two versions, I suspect that this is a bug in gcc's checks for the existence of features in the system headers rather than a problem with the Portfile or patches, but would have to test-build the port to see.

[ThomasDickey (Thomas Dickey)]

Building gcc13 (and comparing with gcc12 build log) from source shows that its configure scripts make several checks for limits.h, failing on the next to last check in gcc13. The relevant chunk of build log:

checking if /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc13/gcc13/work/build/./gcc/xgcc -B/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc13/gcc13/work/build/./gcc/ -B/opt/local/arm64-apple-darwin23/bin/ -B/opt/local/arm64-apple-darwin23/lib/ -isystem /opt/local/arm64-apple-darwin23/include -isystem /opt/local/arm64-apple-darwin23/sys-include   -fchecking=1 supports -fno-rtti -fno-exceptions... checking for unistd.h... yes
checking stdio.h presence... no
checking for -exported_symbols_list linker flag... yes
checking for feof_unlocked... yes
checking for -force_load linker flag... yes
checking for stdio.h... yes
checking limits.h usability... no
checking for /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc13/gcc13/work/build/./gcc/xgcc -B/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc13/gcc13/work/build/./gcc/ -B/opt/local/arm64-apple-darwin23/bin/ -B/opt/local/arm64-apple-darwin23/lib/ -isystem /opt/local/arm64-apple-darwin23/include -isystem /opt/local/arm64-apple-darwin23/sys-include   -fchecking=1 option to produce PIC... -fno-common -DPIC
checking if /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc13/gcc13/work/build/./gcc/xgcc -B/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc13/gcc13/work/build/./gcc/ -B/opt/local/arm64-apple-darwin23/bin/ -B/opt/local/arm64-apple-darwin23/lib/ -isystem /opt/local/arm64-apple-darwin23/include -isystem /opt/local/arm64-apple-darwin23/sys-include   -fchecking=1 PIC flag -fno-common -DPIC works... yes
yes
checking for sys/types.h... checking for dlfcn.h... config.status: creating Makefile
yes
checking if /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc13/gcc13/work/build/./gcc/xgcc -B/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc13/gcc13/work/build/./gcc/ -B/opt/local/arm64-apple-darwin23/bin/ -B/opt/local/arm64-apple-darwin23/lib/ -isystem /opt/local/arm64-apple-darwin23/include -isystem /opt/local/arm64-apple-darwin23/sys-include   -fchecking=1 static flag -static works... yes
checking limits.h presence... config.status: creating config.h
yes

[ThomasDickey (Thomas Dickey)]

Those messages come from the build log in the section beginning

Checking multilib configuration for libitm...                                   
mkdir arm64-apple-darwin23/libitm                                               
Configuring in arm64-apple-darwin23/libitm