cyrus-sasl2: Missing universal variant

[simpplrjay]

It's trying to install for x86

Error: Cannot install kdepimlibs4 for the arch 'x86_64' because
Error: its dependency cyrus-sasl2 is only installed for the arch 'arm64'
Error: and does not have a universal variant.
Error: Unable to execute port: architecture mismatch
--->  Some of the ports you installed have notes:
  db48 has the following notes:
    The Java and Tcl bindings are now provided by the db48-java and
    db48-tcl subports.

[ryandesign (Ryan Carsten Schmidt)]

Yes, MacPorts is trying to install kdepimlibs4 for x86_64 because, like all software depending on qt4-mac, kdepimlibs4 cannot be compiled for arm64. Therefore, MacPorts must install its dependencies universal first.

It's failing because it claims cyrus-sasl2 doesn't have a universal variant, which is surprising because the cyrus-sasl2 Portfile includes the muniversal portgroup so clearly it was intended for it to have a universal variant. There's even a ten-year-old ticket about a problem using cyrus-sasl2's universal variant so it existed at one point.

On my Monterey x86_64 machine, debug output says:

DEBUG: muniversal: < 2 archs supported, not adding universal variant

but I'm not sure why it would say that. The universal_archs in my macports.conf, and I suspect in yours as well, is commented out and therefore set to the default value of arm64 x86_64 which by my count is not less than two. Maybe on my system the muniversal portgroup is concluding that an x86_64 machine cannot run arm64 software (the ability to run the compiled software is sometimes a prerequisite to be able to build it) and is therefore removing the arm64 architecture from the list, leaving just one. But since you have cyrus-sasl2 installed for arm64, I assume you are on an arm64 machine, which can run x86_64 software, so the muniversal portgroup should not have reached that conclusion on your system. Just to make sure, can you run port variants cyrus-sasl2 and see if universal is among those listed? If not, we need to figure out what has gone wrong in the Portfile or portgroup programming.

I see that the cyrus-sasl2 Portfile includes the muniversal portgroup before the legacysupport portgroup whereas we usually include portgroups alphabetically. Sometimes portgroups don't work right when included in the wrong order. The legacysupport portgroup in particular contains code that will complain if it is included after the makefile portgroup because that combination in that order is known to cause problems so we probably need to change the order. However, changing the order does not cause the missing variant to appear for me so something else is also wrong.

[kencu (Ken)]

The cyrus-sasl2 Portfile indicates:

merger_must_run_binaries    yes

so indeed the port can't be built universal x86_64/arm64 on an x86_64 system as that system can't run arm64 binaries.

On my arm64 Sonoma system I had no troubles building cyrus-sasl2 +universal:

% port -v installed cyrus-sasl2
The following ports are currently installed:
  cyrus-sasl2 @2.1.28_1+kerberos+universal (active) requested_variants='+universal' platform='darwin 23' archs='arm64 x86_64' date='2023-11-12T21:53:56-0800'

didn't touch a thing in any Portfiles.

[kencu (Ken)]

by any chance, did you add

-universal

to your variants.conf file?

[jmroot (Joshua Root)]

Setting universal_archs to a value containing less than 2 of the archs supported by the SDK would also do it. That includes an empty value as well as things like i386 x86_64on macOS 11+.

[kencu (Ken)]

installing rosetta is an extra step on an arm64 system.

I wonder if you might see this if rosetta hasn’t been installed yet.

[simpplrjay]

/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_boost176/boost176/main.log

[simpplrjay]

rosetta was installed. didn't touch variants.conf

$ port -v installed cyrus-sasl2

universal wasn't installed

$ sudo port -v install cyrus-sasl2 +universal

worked!

$ port -v installed cyrus-sasl2

The following ports are currently installed:
  cyrus-sasl2 @2.1.28_1+kerberos requested_variants='' platform='darwin 22' archs='arm64' date='2023-11-12T14:34:04-0800'
  cyrus-sasl2 @2.1.28_1+kerberos+universal (active) requested_variants='+universal' platform='darwin 22' archs='arm64 x86_64' date='2023-11-13T14:40:13-0800'

Cool!

$ sudo port -v install kdepimlibs4 +universal

fingers crossed...

Building the Boost C++ Libraries.


error: No best alternative for libs/context/build/asm_sources
    next alternative: required properties: <abi>aapcs <address-model>32 <architecture>arm <binary-format>elf <threading>multi <toolset>clang
        not matched

[..snip..]

error: at libs/thread/build/Jamfile.v2:273
error: 64-bit PPC compilation is not supported when targeting OSX 10.6 or later

Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_boost176/boost176/work/boost_1_76_0" && /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_boost176/boost176/work/boost_1_76_0/b2 -d2 --layout=tagged --debug-configuration --user-config=user-config.jam -sBZIP2_INCLUDE=/opt/local/include -sBZIP2_LIBPATH=/opt/local/lib -sEXPAT_INCLUDE=/opt/local/include -sEXPAT_LIBPATH=/opt/local/lib -sZLIB_INCLUDE=/opt/local/include -sZLIB_LIBPATH=/opt/local/lib -sICU_PATH=/opt/local variant=release runtime-link=shared -j8 --no-cmake-config link=shared threading=multi pch=off address-model=64 architecture=combined
Exit code: 1
Error: Failed to build boost176: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_boost176/boost176/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
Error: Processing of port kdepimlibs4 failed
--->  Some of the ports you installed have notes:
  lzma has the following notes:
    The LZMA SDK program is installed as "lzma_alone", to avoid conflict with LZMA Utils

ugh.

[kencu (Ken)]

serious progress. still odd how you would get this:

Error: its dependency cyrus-sasl2 is only installed for the arch 'arm64'
Error: and does not have a universal variant.

and yet somehow this worked anyway:

sudo port -v install cyrus-sasl2 +universal

for that, I have no explanation for you. But moving on:

Instead of trying to install kdepimlibs4 +universal, which I don't think can ever work, please just try this:

sudo port clean kdepimlibs4
sudo port clean boost176

sudo port -v install kdepimlibs4

[simpplrjay]

well, we go for "progress not perfection" around here.

tried it. same result.

error: at libs/thread/build/Jamfile.v2:273
error: 64-bit PPC compilation is not supported when targeting OSX 10.6 or later

Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_boost176/boost176/work/boost_1_76_0" && /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_boost176/boost176/work/boost_1_76_0/b2 -d2 --layout=tagged --debug-configuration --user-config=user-config.jam -sBZIP2_INCLUDE=/opt/local/include -sBZIP2_LIBPATH=/opt/local/lib -sEXPAT_INCLUDE=/opt/local/include -sEXPAT_LIBPATH=/opt/local/lib -sZLIB_INCLUDE=/opt/local/include -sZLIB_LIBPATH=/opt/local/lib -sICU_PATH=/opt/local variant=release runtime-link=shared -j8 --no-cmake-config link=shared threading=multi pch=off address-model=64 architecture=combined
Exit code: 1
Error: Failed to build boost176: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_boost176/boost176/main.log for details.
Error: Unable to execute port: upgrade boost176 failed

[kencu (Ken)]

well, how about we close this ticket as you have (somehow) built cyrus-sasl2 as +universal, so it's no longer missing a universal variant for you.

If you are having a problem building boost176, we can start a fresh ticket for that one, with a new log, so we don't get confused about this ticket.

I may build kde again on my M1, just to see if something has changed since I built it last year. I take it you are wanting to install kdepimlibs4 on your M1? Is that your final goal? Or is there some kde port you're trying for?

(I installed Debian 12 on my M1 using UTM, and then installed KDE on that, and it works just great, by the way, super fast as it's all arm64, free, and also it uses a newer KDE using QT5. But I digress).

[ryandesign (Ryan Carsten Schmidt)]

Replying to simpplrjay:

error: 64-bit PPC compilation is not supported when targeting OSX 10.6 or later

Looks like #64954 to me.

[kencu (Ken)]

nope -- there is something weird going on here, for sure.

I installed cmake-devel and then:

% sudo port -v install kdepimlibs4 
--->  Computing dependencies for kdepimlibs4.
Error: Cannot install kdepimlibs4 for the arch 'x86_64' because
Error: its dependency cyrus-sasl2 does not build for the required arch by default
Error: and does not have a universal variant.
Error: Follow https://guide.macports.org/#project.tickets if you believe there
is a bug.
Error: Processing of port kdepimlibs4 failed

[kencu (Ken)]

and I get the same error even when starting with no ports.

heh. A puzzle.

[kencu (Ken)]

I ran into the same boost176 error.

I last built the kde4 ports (all of them I recall) on an arm64 system in February, but it appears things have changed in some way, and it will need to be looked at again.

[jmroot (Joshua Root)]

My guess would be that cyrus-sasl2 is not listed as having a universal variant in the PortIndex.

[simpplrjay]

Replying to kencu:

If you are having a problem building boost176, we can start a fresh ticket for that one, with a new log, so we don't get confused about this ticket.

That might be a good idea. Or move this discussion to #64954

I may build kde again on my M1, just to see if something has changed since I built it last year. I take it you are wanting to install kdepimlibs4 on your M1? Is that your final goal? Or is there some kde port you're trying for?

ktouch
​https://ports.macports.org/port/ktouch/details/

(I installed Debian 12 on my M1 using UTM, and then installed KDE on that, and it works just great, by the way, super fast as it's all arm64, free, and also it uses a newer KDE using QT5. But I digress).

I have been known to digress. It's a common trait.
Thanks for your help, Ken and everyone on this ticket!

[kencu (Ken)]

something strange is going on here.

Josh has a clue that the index was built on an Intel system but is being used on an arm64 system, so it is -- wrong, basically.

The last time I built kde4 I know I used a custom index I made on my local machine.

So that is the next thing I plan to test.

[kencu (Ken)]

using a locally-made portindex indeed fixes the mysterious "cyrus-sasl2 has no universal variant" issue.

what the general fix for this issue in in MacPorts I don't know.

Then boost176 is a separate deal, as per #64954, and still needs a proper fix.

[ryandesign (Ryan Carsten Schmidt)]

The mprsyncup script​ that generates the portindexes on the server attempts to convince MacPorts that it is running on other OS versions and architectures so that those indexes will be correct for those other OS versions and architectures. In this instance, something has evidently gone wrong. I see that all of the portindexes for i386 list cyrus-sasl2 as having a universal variant and none of the portindexes for arm or powerpc do. The machine that runs mprsyncup is running OS X 10.11.

For example, mprsyncup runs portindex -p macosx_22_arm when generating the index for Ventura for Apple Silicon. portindex​ splits this value on _ and uses the components to set os.platform, os.major, and os.arch, respectively. (The next version of MacPorts will also set os.version.) When merger_must_run_binaries is yes, muniversal-1.0.tcl​ uses os.arch in its decision about which archs to exclude. So I am not sure yet what has gone wrong.

[kencu (Ken)]

has duplicate #68899

[kencu (Ken)]

so the direct way to test/fix this might be to have a custom repo with only cyrus-sasl2 in it, and run portindex using that arm forcer on 10.11:

portindex -p macosx_22_arm

and see what comes out.

[kencu (Ken)]

my first guess is that using sysctl hw.cpu64bit_capable in the muniversal PortGroup doesn't work right when spoofing the os you are running on:

​https://github.com/macports/macports-ports/blob/46b48388599a7e07428c47fa9e527c058ce7c199/_resources/port1.0/group/muniversal-1.0.tcl#L121

[jmroot (Joshua Root)]

Replying to kencu:

my first guess is that using sysctl hw.cpu64bit_capable in the muniversal PortGroup doesn't work right when you are spoofing the os you are running on.

Yes, that will of course always reflect the hardware that you are actually running on at the time.

[jmroot (Joshua Root)]

In 07972a956df49c6998722bcf783ba5aba47d1ec1/macports-ports (master):

muniversal: avoid sysctl where possible

See: #68685

[jmroot (Joshua Root)]

I don't think that will actually help here, since muniversal is still deciding whether to create the variant based on the universal_archs.

[jmroot (Joshua Root)]

In a8b8426d769aea7e2a99e188ee8d8bf30e0d5b82/macports-ports (master):

muniversal-1.0: be less eager to avoid universal variant

Closes: #68685

[ryandesign (Ryan Carsten Schmidt)]

It doesn't appear to be fixed, given the filing of duplicate #69475. Looking at the PortIndexes on the server, cyrus-sasl2 still only has a universal variant in the i386 indexes.

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #69856.