#70764 closed defect (fixed)
Segfault fetching ports from FTP URLs
Reported by: | Ygar-Vanaxe-Tsy-xtledron (YVT) | Owned by: | jmroot (Joshua Root) |
---|---|---|---|
Priority: | Normal | Milestone: | MacPorts 2.10.2 |
Component: | base | Version: | 2.10.1 |
Keywords: | monterey, ventura, sonoma, sequoia | Cc: | Ygar-Vanaxe-Tsy-xtledron (YVT), jmroot (Joshua Root), wmszeliga, ryandesign (Ryan Carsten Schmidt), hapaguy (Brian Kurt Fujikawa), Dave-Allured (Dave Allured) |
Port: |
Description (last modified by Ygar-Vanaxe-Tsy-xtledron (YVT))
When migrating to darwin 24, restore command failed due to ncurses seg fault.
$ sudo port install ncurses ---> Fetching archive for ncurses ---> Attempting to fetch ncurses-6.5_0.darwin_24.arm64.tbz2 from https://packages.macports.org/ncurses ---> Attempting to fetch ncurses-6.5_0.darwin_24.arm64.tbz2 from https://cph.dk.packages.macports.org/ncurses ---> Attempting to fetch ncurses-6.5_0.darwin_24.arm64.tbz2 from https://mse.uk.packages.macports.org/ncurses ---> Fetching distfiles for ncurses ---> Attempting to fetch ncurses-6.5.tar.gz from ftp://ftp.funet.fi/pub/gnu/prep/ncurses [1] 29141 segmentation fault sudo port install ncurses
As of earlier today, ncurses installs without issue. However, the same problem now occurs with psutils.
---> Computing dependencies for psutils ---> Fetching archive for psutils ---> Attempting to fetch psutils-p17_1.darwin_24.arm64.tbz2 from https://packages.macports.org/psutils ---> Attempting to fetch psutils-p17_1.darwin_24.arm64.tbz2 from https://cph.dk.packages.macports.org/psutils ---> Attempting to fetch psutils-p17_1.darwin_24.arm64.tbz2 from https://mse.uk.packages.macports.org/psutils ---> Fetching distfiles for psutils ---> Attempting to fetch psutils-p17.tar.gz from ftp://mirror.macomnet.net/pub/CTAN/support/psutils [1] 92420 segmentation fault sudo port install psutils
Or sometimes due to bus error.
---> Computing dependencies for psutils ---> Fetching archive for psutils ---> Attempting to fetch psutils-p17_1.darwin_24.arm64.tbz2 from https://packages.macports.org/psutils ---> Attempting to fetch psutils-p17_1.darwin_24.arm64.tbz2 from https://cph.dk.packages.macports.org/psutils ---> Attempting to fetch psutils-p17_1.darwin_24.arm64.tbz2 from https://mse.uk.packages.macports.org/psutils ---> Fetching distfiles for psutils ---> Attempting to fetch psutils-p17.tar.gz from ftp://mirror.macomnet.net/pub/CTAN/support/psutils [1] 92223 bus error sudo port install psutils
Attachments (1)
Change History (28)
comment:1 Changed 2 months ago by Ygar-Vanaxe-Tsy-xtledron (YVT)
Keywords: | segfault, sequoia, ,migration → segfault, sequoia, migration |
---|
comment:2 Changed 2 months ago by Ygar-Vanaxe-Tsy-xtledron (YVT)
Keywords: | ncurses added |
---|---|
Priority: | Normal → High |
comment:3 Changed 2 months ago by Ygar-Vanaxe-Tsy-xtledron (YVT)
Cc: | Ygar-Vanaxe-Tsy-xtledron added |
---|
comment:4 Changed 2 months ago by jmroot (Joshua Root)
Description: | modified (diff) |
---|---|
Keywords: | segfault ncurses migration removed |
Summary: | ncurses Segfault on MacOS 15.0 → Segfault fetching ncurses on MacOS 15.0 |
comment:5 follow-up: 8 Changed 8 weeks ago by Ygar-Vanaxe-Tsy-xtledron (YVT)
Nothing crashed despite the segfault, ncurses simply failed to install making its dependents (pandoc for example) also un-installable. As of writing, ncurses installs fine.
sudo port install ncurses ---> Fetching archive for ncurses ---> Attempting to fetch ncurses-6.5_0.darwin_24.arm64.tbz2 from https://packages.macports.org/ncurses ---> Attempting to fetch ncurses-6.5_0.darwin_24.arm64.tbz2 from https://cph.dk.packages.macports.org/ncurses ---> Attempting to fetch ncurses-6.5_0.darwin_24.arm64.tbz2 from https://mse.uk.packages.macports.org/ncurses ---> Fetching distfiles for ncurses ---> Attempting to fetch ncurses-6.5.tar.gz from https://fra.de.distfiles.macports.org/ncurses ---> Verifying checksums for ncurses ---> Extracting ncurses ---> Applying patches to ncurses ---> Configuring ncurses ---> Building ncurses ---> Staging ncurses into destroot ---> Installing ncurses @6.5_0 ---> Activating ncurses @6.5_0 ---> Cleaning ncurses ---> Updating database of binaries ---> Scanning binaries for linking errors ---> No broken files found. ---> No broken ports found.
comment:6 Changed 8 weeks ago by Ygar-Vanaxe-Tsy-xtledron (YVT)
Description: | modified (diff) |
---|---|
Priority: | High → Normal |
Summary: | Segfault fetching ncurses on MacOS 15.0 → Segfault fetching ports on MacOS 15.0 |
comment:7 Changed 8 weeks ago by Ygar-Vanaxe-Tsy-xtledron (YVT)
Latest attempt to install outputs this:
sudo port install psutils ---> Computing dependencies for psutils ---> Fetching archive for psutils ---> Attempting to fetch psutils-p17_1.darwin_24.arm64.tbz2 from https://packages.macports.org/psutils ---> Attempting to fetch psutils-p17_1.darwin_24.arm64.tbz2 from https://cph.dk.packages.macports.org/psutils ---> Attempting to fetch psutils-p17_1.darwin_24.arm64.tbz2 from https://mse.uk.packages.macports.org/psutils ---> Fetching distfiles for psutils ---> Attempting to fetch psutils-p17.tar.gz from ftp://mirror.macomnet.net/pub/CTAN/support/psutils tclsh8.6(93125,0x1f3f77240) malloc: *** error for object 0x600001384081: pointer being freed was not allocated tclsh8.6(93125,0x1f3f77240) malloc: *** set a breakpoint in malloc_error_break to debug [1] 93124 abort sudo port install psutils
comment:8 Changed 8 weeks ago by ryandesign (Ryan Carsten Schmidt)
Replying to Ygar-Vanaxe-Tsy-xtledron:
Nothing crashed despite the segfault
A segfault is a crash. Please double-check whether macOS has recorded a crash log and if so please attach it; without it it's difficult for anyone to diagnose where in the code the problem needs to be fixed.
comment:10 Changed 8 weeks ago by jmroot (Joshua Root)
Cc: | jmroot added |
---|---|
Component: | ports → base |
Port: | ncurses removed |
Summary: | Segfault fetching ports on MacOS 15.0 → Segfault fetching ports from FTP URLs on MacOS 15.0 |
comment:11 Changed 8 weeks ago by wmszeliga
I'm having the same issue with installing the psutils port. It appears as if tclch8.6 is the program crashing. Here's the relevant thread info from the crash log.
Termination Reason: Namespace SIGNAL, Code 11 Segmentation fault: 11 Terminating Process: exc handler [48902] VM Region Info: 0x892513e3ad is not in any region. Bytes after previous region: 107996242862 Bytes before following region: 104964083686483 REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL commpage (reserved) 1000000000-7000000000 [384.0G] ---/--- SM=NUL reserved VM address space (unallocated) ---> GAP OF 0x5f9000000000 BYTES MALLOC_NANO 600000000000-600020000000 [512.0M] rw-/rwx SM=PRV Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libcurl.4.dylib 0x1b53e27b4 Curl_bufq_free + 36 1 libcurl.4.dylib 0x1b5401134 http2_data_done + 216 2 libcurl.4.dylib 0x1b53fe2d4 cf_h2_cntrl + 108 3 libcurl.4.dylib 0x1b53dafb0 Curl_conn_ev_data_detach + 104 4 libcurl.4.dylib 0x1b540fc2c Curl_detach_connection + 40 5 libcurl.4.dylib 0x1b542d968 extract_if_dead + 376 6 libcurl.4.dylib 0x1b542d7d8 call_extract_if_dead + 36 7 libcurl.4.dylib 0x1b53dba98 Curl_conncache_foreach + 152 8 libcurl.4.dylib 0x1b542c324 prune_dead_connections + 152 9 libcurl.4.dylib 0x1b542b008 Curl_connect + 4420 10 libcurl.4.dylib 0x1b5410904 multi_runsingle + 476 11 libcurl.4.dylib 0x1b541067c curl_multi_perform + 208 12 Pextlib.dylib 0x102b727d8 CurlFetchCmd + 2256 13 Pextlib.dylib 0x102b73c60 CurlCmd + 212 14 libtcl8.6.dylib 0x102d752b8 TclNRRunCallbacks + 80 15 libtcl8.6.dylib 0x102e00778 ChildEval + 132 16 libtcl8.6.dylib 0x102e0211c NRChildCmd + 744 17 libtcl8.6.dylib 0x102d752b8 TclNRRunCallbacks + 80 18 libtcl8.6.dylib 0x102d7603c TclEvalEx + 1668 19 libtcl8.6.dylib 0x102e148cc Tcl_FSEvalFileEx + 512 20 libtcl8.6.dylib 0x102e1b428 Tcl_MainEx + 920 21 tclsh8.6 0x102a83a4c main + 48 22 dyld 0x19c304274 start + 2840
comment:12 follow-up: 14 Changed 8 weeks ago by jmroot (Joshua Root)
Cc: | wmszeliga added |
---|
OK, so it's crashing pretty deep in libcurl calls, which means it's probably an Apple bug. The question is, how can we mitigate it?
@wmszeliga, was your crash also when fetching from an FTP site? If so, it's a bit odd that http2_data_done
is involved.
comment:13 Changed 8 weeks ago by jmroot (Joshua Root)
The workarounds from ProblemHotlist#fetch-failures should work here as well. You could also try editing macports.conf
and adding distfiles.macports.org
or some other http mirror to preferred_hosts
.
comment:14 Changed 8 weeks ago by ryandesign (Ryan Carsten Schmidt)
Replying to jmroot:
OK, so it's crashing pretty deep in libcurl calls, which means it's probably an Apple bug.
Not necessarily. We may be calling curl incorrectly in a way that causes the crash. However I see macOS 15.0 has curl 8.7.1 and this looks like https://github.com/curl/curl/issues/13731 which was fixed in curl 8.8.0.
I tried /usr/bin/curl -OL ftp://ftp.funet.fi/pub/gnu/prep/ncurses/ncurses-6.5.tar.gz
on macOS 15 and it downloaded successfully without crashing so it is possible to download from ftp without crashing. The upstream bug report suggests the crash occurs when a curl easy handle that had been used for http2 is then reused for a different protocol. The fix would be not reusing curl easy handles.
comment:15 Changed 8 weeks ago by ryandesign (Ryan Carsten Schmidt)
Keywords: | monterey ventura sonoma added |
---|---|
Summary: | Segfault fetching ports from FTP URLs on MacOS 15.0 → Segfault fetching ports from FTP URLs |
macOS 12.7.6, macOS 13.7, and macOS 14.7 also shipped with curl 8.7.1 and would also be affected but users probably notice it less or not at all since we have binaries for those OS versions. macOS 11.7.10 has curl 8.1.2 and is not affected.
comment:16 Changed 8 weeks ago by jmroot (Joshua Root)
Owner: | set to jmroot |
---|---|
Resolution: | → fixed |
Status: | new → closed |
comment:17 Changed 8 weeks ago by ryandesign (Ryan Carsten Schmidt)
Cc: | ryandesign added |
---|---|
Milestone: | → MacPorts Future |
Thanks Josh. Hopefully we can get Apple to ship fixed versions of curl. I was trying to create a minimal example that causes a crash that I could include in a bug report to Apple but so far my example does not crash.
comment:18 Changed 8 weeks ago by jmroot (Joshua Root)
It seems like you often need multiple reuses of a handle to get it to the point where it actually crashes. The most minimal example I found so far was port -v livecheck libsdl2_net libsdl2_ttf postfix
.
comment:19 Changed 8 weeks ago by ryandesign (Ryan Carsten Schmidt)
I meant a minimal example not requiring MacPorts. I wrote a C program to do an http2 request and then an ftp request and it succeeded.
Your fix dumps and reinitializes the handle if the scheme changes, but the upstream bug report didn't talk about schemes; it talked about protocols. I suspect the crash would occur if accessing an http2 host and then accessing an http1 host but I would like to test that in a minimal example program.
comment:20 Changed 8 weeks ago by jmroot (Joshua Root)
Based on the above, this works as a reproducer using only /usr/bin/curl:
/usr/bin/curl -o 1.txt -o 2.txt -o 3.txt https://github.com/libsdl-org/SDL_net/tags https://github.com/libsdl-org/SDL_ttf/tags ftp://ftp.porcupine.org/mirrors/postfix-release/official/
comment:21 Changed 8 weeks ago by jmroot (Joshua Root)
Replacing the third URL in that example with http://www.iozone.org/
, which is a HTTP 1.1 site, doesn't crash.
comment:22 Changed 8 weeks ago by jmroot (Joshua Root)
The fix I committed stopped the crash for livecheck, but I've since seen the same crash while fetching a port. Maybe making a new handle if either the scheme or the host change would work, and allow us to keep the performance benefits of reusing the handle for subsequent requests to the same host.
comment:23 Changed 8 weeks ago by jmroot (Joshua Root)
comment:24 Changed 8 weeks ago by jmroot (Joshua Root)
That seems to do the trick so far. Note that I found that reallocating the multi handle, not the easy handle, was necessary. That doesn't seem to be what the upstream discussion was implying, so maybe it would be a good idea to follow up there about whether we actually need to reallocate both handles.
comment:25 Changed 7 weeks ago by jmroot (Joshua Root)
Milestone: | MacPorts Future → MacPorts 2.10.2 |
---|
comment:26 Changed 7 weeks ago by hapaguy (Brian Kurt Fujikawa)
Cc: | hapaguy added |
---|
comment:27 Changed 7 weeks ago by Dave-Allured (Dave Allured)
Cc: | Dave-Allured added |
---|
Can you attach the crash report?