Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#66163 closed defect (fixed)

muniversal does not work properly with the new /usr/bin/diff that ships with Ventura

Reported by: Zeta-0x00 (Z) Owned by: MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
Priority: Normal Milestone:
Component: ports Version: 2.8.0
Keywords: ventura Cc: xroblot, chrisminett (Chris Minett), radarhere (Andrew Murray)
Port: gmp

Description (last modified by kencu (Ken))

Trying to install gdb always report the same error, i've tried to reinstall every dependency especially the gmp but it doesn't work, also I removed gmp and every directory path, but the same error

error """""""""""""""

--->  Staging gmp into destroot
Error: Failed to destroot gmp: error copying "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmp/gmp/work/destroot" to "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmp/gmp/work/destroot-arm64/destroot": file already exists
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmp/gmp/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets
if you believe there is a bug.
Error: Processing of port gdb failed

Attachments (3)

main.log (1.1 MB) - added by greyhare 2 years ago.
main.log
work.tar.gz (116.5 KB) - added by greyhare 2 years ago.
Various gmp.h files in a tar/gzip archive.
patch-muniversal-ventura.diff (2.9 KB) - added by kencu (Ken) 2 years ago.

Download all attachments as: .zip

Change History (31)

comment:1 Changed 2 years ago by kencu (Ken)

Description: modified (diff)
Keywords: gmp destrroot arm64 gdb removed
Port: gmp added; gdb removed
Priority: HighNormal
Summary: GDB can't be installed in macOS 13 Ventura (ARM64)gmp fails to destroot on macOS 13 Ventura (ARM64): file already exists

please attach the main.log so we can assist you

please try this:

sudo port clean gmp
sudo port -v install gmp

It looks like you might be trying to build something +universal….is that right?

comment:2 Changed 2 years ago by xroblot

Cc: xroblot added

comment:3 Changed 2 years ago by xroblot

I have a similar error and I was indeed trying to build gmp with +universal on a M1 MacBook Air / macOS 13 Ventura

Last edited 2 years ago by xroblot (previous) (diff)

comment:4 Changed 2 years ago by chrisminett (Chris Minett)

Cc: chrisminett added

comment:5 Changed 2 years ago by greyhare

I'm having the same problem on Ventura on a Mac Studio (M1 Ultra or whatever got me all the cores) and I'm not using any variants.

Changed 2 years ago by greyhare

Attachment: main.log added

main.log

comment:6 Changed 2 years ago by greyhare

Hmmm...

$ sudo port -q install [local package]
--->  Computing dependencies for [local package]
--->  Dependencies to be installed: gdb gmp libiconv ncurses zlib glide go libev mercurial py310-re2 re2 nmap npm8 nodejs16 openssl11 subversion apr apr-util cyrus-sasl2 dbus libutf8proc serf1 scons
--->  Fetching archive for gmp
--->  Attempting to fetch gmp-6.2.1_1+universal.darwin_22.arm64-x86_64.tbz2 from https://packages.macports.org/gmp
--->  Attempting to fetch gmp-6.2.1_1+universal.darwin_22.arm64-x86_64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/gmp
--->  Attempting to fetch gmp-6.2.1_1+universal.darwin_22.arm64-x86_64.tbz2 from https://mse.uk.packages.macports.org/gmp
Error: Failed to destroot gmp: /opt/local/include/gmp.h differs in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmp/gmp/work/destroot-arm64 and /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmp/gmp/work/destroot-ppc-intel and cannot be merged
$ sudo port clean gmp
--->  Cleaning gmp
$ sudo port -v install gmp
--->  Cleaning gmp
--->  Removing work directory for gmp
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.

It looks like I already had a non-universal gmp, and something that needs gmp (one of gdb libiconv ncurses zlib glide go libev mercurial py310-re2 re2 nmap npm8 nodejs16 openssl11 subversion apr apr-util cyrus-sasl2 dbus libutf8proc serf1 scons) is adding the +universal.

I'm not sure what the command is for "find deep dependency in uninstalled package X that pulls in package Y".

comment:7 Changed 2 years ago by greyhare

Direct deps for that local package I'm trying to install:

depends_run                     port:boost port:cargo port:cctools port:clang-14 \
                                        port:cmake port:desimone-base port:desimone-python \
                                        port:gcc12 port:gdb port:git port:glide port:gmake \
                                        port:go port:libev port:libusb port:mercurial \
                                        port:ninja port:nmap port:npm8 port:perl5 port:rust \
                                        port:subversion

comment:8 Changed 2 years ago by ryandesign (Ryan Carsten Schmidt)

Please attach the two gmp.h files so that we can see how they differ.

comment:9 Changed 2 years ago by ryandesign (Ryan Carsten Schmidt)

Keywords: ventura added
Owner: set to MarcusCalhoun-Lopez
Status: newassigned
Summary: gmp fails to destroot on macOS 13 Ventura (ARM64): file already existsgmp @6.2.1+universal: gmp.h differs ... and cannot be merged

comment:10 Changed 2 years ago by greyhare

OK, I'll try to find them. BTW, gdb is what's depending on gmp+universal.

Changed 2 years ago by greyhare

Attachment: work.tar.gz added

Various gmp.h files in a tar/gzip archive.

comment:11 Changed 2 years ago by greyhare

OK, attached. Almost all these came from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmp/gmp/work. The exception is ./work/native-gmp-that-worked/gmp.h which I copied from /opt/local/include/gmp.h which was installed by the non-+universal install of gmp that worked.

./work/destroot-arm64/opt/local/include/gmp.h
./work/gmp-6.2.1-x86_64/gmp.h
./work/gmp-6.2.1-arm64/gmp.h
./work/destroot-ppc-intel/opt/local/include/gmp.h
./work/native-gmp-that-worked/gmp.h
./work/destroot-intel/opt/local/include/gmp.h
./work/destroot-x86_64/opt/local/include/gmp.h

comment:12 Changed 2 years ago by kencu (Ken)

On Monterey and less the muniversal PG does the right thing with the differences automatically:

/* Define CC and CFLAGS which were used to build this version of GMP */
#ifdef __arm64__
#define __GMP_CC "/usr/bin/clang -arch arm64"
#define __GMP_CFLAGS "-O2 -pedantic -march=armv8-a"
#else
#define __GMP_CC "/usr/bin/clang -arch x86_64"
#define __GMP_CFLAGS "-O2 -pedantic -fomit-frame-pointer -m64 -mtune=core2 -march=core2"
#endif

but on Ventura the new diff seems to be failing to do this. That is exactly what the no-longer-accepted flag was supposed to do:

`--old-group-format=#ifdef __arm64__

66103

comment:13 Changed 2 years ago by kencu (Ken)

Next step if it was me would be to install diffutils, edit the muniversal PG to use that diff, and see if that fixes it. If it does, then at least we know exactly what is going on, and can work on how to fix it (using diffutils diff, it would seem).

comment:14 Changed 2 years ago by Zeta-0x00 (Z)

Didn't work, ~$ sudo port install gdb --old-group-format=#ifdef arm64 ---> Fetching archive for gmp ---> Attempting to fetch gmp-6.2.1_1+universal.darwin_22.arm64-x86_64.tbz2 from https://packages.macports.org/gmp ---> Attempting to fetch gmp-6.2.1_1+universal.darwin_22.arm64-x86_64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/gmp ---> Attempting to fetch gmp-6.2.1_1+universal.darwin_22.arm64-x86_64.tbz2 from https://mse.uk.packages.macports.org/gmp ---> Fetching distfiles for gmp ---> Verifying checksums for gmp ---> Extracting gmp ---> Applying patches to gmp ---> Applying patches to gmp ---> Configuring gmp ---> Building gmp ---> Staging gmp into destroot Error: Failed to destroot gmp: /opt/local/include/gmp.h differs in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmp/gmp/work/destroot-arm64 and /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmp/gmp/work/destroot-ppc-intel and cannot be merged Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmp/gmp/main.log for details. Error: Unable to execute port: upgrade gmp failed

comment:15 Changed 2 years ago by kencu (Ken)

Oh, my apologies, that is not quite what I meant. It's a bit complicated to explain, so sorry. I think I might just have to try it myself.

comment:16 Changed 2 years ago by kencu (Ken)

yeah, that's it:

% port -v installed gmp
The following ports are currently installed:
  gmp @6.2.1_1+universal (active) requested_variants='+universal' platform='darwin 22' archs='arm64 x86_64' date='2022-11-09T23:32:34-0800'

Unless somebody comes up with a better plan, for now you must install diffutils and then change muniversal to use it, by substituting ${prefix}/bin/gdiff for /usr/bin/diff. I'll post up the diff I used, for short term fixes. But the definitive fix will need some pondering, as per the other ticket.

Changed 2 years ago by kencu (Ken)

comment:17 Changed 2 years ago by kencu (Ken)

Summary: gmp @6.2.1+universal: gmp.h differs ... and cannot be mergedmuniversal does not work properly with the new /usr/bin/diff that ships with Ventura

MacPorts might need to bundle a diff (and patch?) that work

comment:18 Changed 2 years ago by radarhere (Andrew Murray)

Cc: radarhere added

comment:19 in reply to:  description Changed 2 years ago by plk

I tried the patch for muniversal and cleaned/rebuilt libffi - same issue however - is there something else that needs to be done to do a temporary workaround - everything stops with libffi failing to stage due to the diff issue for me.

comment:20 Changed 2 years ago by kencu (Ken)

the patch will be overwritten each time you selfupdate, so you might check that it is applied.

not ideal, but it was just a quick proof-of-concept pending a permanent fix

comment:21 Changed 2 years ago by kencu (Ken)

oh, and there are two muniversal portgroups, btw :)

I only twiddled with the one that libffi uses.

A real fix will depend on Ryan or Josh deciding whether MP will bundle a diff/patch or try to use the one in ${prefix} or install a dedicated one in libexec…

comment:22 Changed 2 years ago by greyhare

Can't build gdb on Ventura/arm64 until they decide.

comment:23 Changed 2 years ago by kencu (Ken)

installing diffutils and editing the two muniversal portgroups takes 30 seconds and you're on your way for the moment, if this is mission-critical for you.

the permanent fix requires some admin decisions.

Should all systems use the new diff, or just Ventura+? Best to have everything the same, but why change something that has been working all these years... so probably just Ventura+.

The current diffutils port has way too many deps due to a reliance on "xz". Can we eliminate all those if we assume we're building on Ventura+, where xz is presumably available and new enough?

% port rdeps diffutils
The following ports are dependencies of diffutils @3.8_0:
  xz
    gettext
      libiconv
        gperf
      libtextstyle
        ncurses
      gettext-runtime
      gettext-tools-libs

Even if xz is removed somehow, we need a sequestered gettext as well to build diffutils, which brings in another ton of deps -- that is a pain. Can we somehow eliminate that dependency, perhaps by using the system libs?

% port rdeps diffutils        
The following ports are dependencies of diffutils @3.8_0:
  gettext
    libiconv
      gperf
    libtextstyle
      ncurses
    gettext-runtime
    gettext-tools-libs

Should we install all of diffutils, or just "diff" which is all we currently require? Just diff would be much smaller and simpler to manage.

comment:24 Changed 2 years ago by kencu (Ken)

here's one bit of good news -- on Ventura, diff builds from the source tarball without any help from any MacPorts' deps:

% otool -L diff
diff:
	/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)

so it can be built in isolation, which simplifies things considerably if we go that route.

Last edited 2 years ago by kencu (Ken) (previous) (diff)

comment:25 Changed 2 years ago by kencu (Ken)

and /usr/bin/tar xJvf diffutils-3.8.tar.xz works just dandy on Ventura to extract the archive without anything from MacPorts as well.

So this looks do-able without too much fuss.

comment:26 Changed 2 years ago by kencu (Ken)

Last edited 2 years ago by kencu (Ken) (previous) (diff)

comment:27 Changed 2 years ago by Ken <21211439+kencu@…>

Resolution: fixed
Status: assignedclosed

In 5dccc228b10aff819500672c6f4203da53849dc3/macports-ports (master):

diffutils-for-muniversal: v3.8 (https://github.com/macports/macports-ports/pull/16642)

this commit adds a sequestered, isolated diff utility for use on Ventura (and above).
It also changes the muniversal PortGroups to use the new diff on Ventura+/

the new diff utility that comes with Ventura does not accept options that the muniversal PortGroup requires to properly function.

closes: #66163
closes: #66103

comment:28 Changed 2 years ago by greyhare

BTW, when I see these, do you guys want to be notified?

Warning: Configuration logfiles contain indications of -Wimplicit-function-declaration; check that features were not accidentally disabled:
  re_search: found in diffutils-3.8/config.log
  re_compile_pattern: found in diffutils-3.8/config.log
  re_set_syntax: found in diffutils-3.8/config.log
  MIN: found in diffutils-3.8/config.log
  strchr: found in diffutils-3.8/config.log
Note: See TracTickets for help on using tickets.