opencv3 has undocumented dependencies

[mouse07410 (Mouse)]

First problem - two darn large dependencies that are not listed: llvm-9.0 and clang-9.0. Second problem - these dependencies are old anyway, and it's about time to update them to, e.g., llvm-13 and clang-13.

And if it merely wants Macports-provided LLVM and Clang - why can't it find what's installed? (llvm-11 and llvm-13, clang-11 and clang-13 on my machine).

[ryandesign (Ryan Carsten Schmidt)]

Please explain why you think opencv3 requires llvm-9.0 and clang-9.0. We have successful builds of this port on all 64-bit OS versions on our automated build system.

What error did you see? If opencv3 failed to build, attach the main.log file.

[mouse07410 (Mouse)]

Please explain why you think opencv3 requires llvm-9.0 and clang-9.0.

Because it demanded to install both of them. Which I did not want, but had to concede. ;-) My machines usually run the latest Macports release of llvm and clang (and gcc), plus maybe an older version that is required for a specific reason (happens rarely).

What error did you see?

No error - just an unnecessary (for me ;) installation of two huge packages (llvm-9.0 and clang-9.0) that I need to remove afterwards anyway.

This ticket is about getting opencv3 to stop pulling and installing llvm-9.0 and clang-9.0 - or documenting that they are required for build-from-source (and in that case - upgrading the "dependency" to llvm-13 and clang-13).

[mascguy (Christopher Nielsen)]

Mouse, just to help us understand: Are you questioning why MacPorts is choosing to build this port with Clang 9...?

[mouse07410 (Mouse)]

Are you questioning why MacPorts is choosing to build this port with Clang 9...?

Basically, yes. Especially, since Macports Clang-9.0 was not installed on my machine (and Macports Clang-13 and Clang-11 were).

There are few ports (AFAIK) that require Macports-provided Clang (e.g., I might want opencv3 or whatever, but have and use only Xcode Clang).

If a port does have a reason to insist on Macports Clang - first, it should be listed as a build dependency, and second - it should be updated to the current (or closer to current) version of Macports Clang. 9.0 is old at this point.

[jmroot (Joshua Root)]

The dependency will show up in port deps and port info if it's used, e.g.

% port deps opencv3
Full Name: opencv3 @3.4.16_1
Build Dependencies:   cmake, pkgconfig, clang-9.0
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg,
                      tiff, webp, ilmbase, openexr, libcxx, legacy-support

That would be the case on systems where the Xcode clang doesn't support C++11. Another possibility is that clang-9.0 wasn't being installed for opencv3 at all, but for a dependency like perhaps openexr. A log would clear all of this up.

[mouse07410 (Mouse)]

$ port info opencv3
opencv3 @3.4.16_1 (graphics, science)
Sub-ports:            py27-opencv3, py35-opencv3, py36-opencv3, py37-opencv3, py38-opencv3, py39-opencv3
Variants:             avx, avx2, contrib, dc1394, debug, eigen, gdal, java, nonfree, opencl, openmp, openni,
                      qt4, qt5, tbb, tests, vtk

Description:          OpenCV (Open Source Computer Vision Library) is an open source computer vision and
                      machine learning software library. OpenCV was built to provide a common infrastructure
                      for computer vision applications and to accelerate the use of machine perception in the
                      commercial products. Being a BSD-licensed product, OpenCV makes it easy for businesses
                      to utilize and modify the code.
Homepage:             https://opencv.org

Build Dependencies:   cmake, pkgconfig
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg, tiff, webp, ilmbase, openexr
Conflicts with:       opencv3-devel
Platforms:            darwin
License:              BSD
Maintainers:          Email: mascguy@macports.org, GitHub: mascguy
$ port info openexr
openexr @2.3.0_2 (graphics)
Sub-ports:            ilmbase, py27-pyilmbase, py35-pyilmbase, py36-pyilmbase, py37-pyilmbase, py38-pyilmbase,
                      py39-pyilmbase, openexr_viewers

Description:          OpenEXR is a high dynamic-range (HDR) image file format developed by Industrial Light &
                      Magic for use in computer imaging applications.
Homepage:             https://www.openexr.com

Build Dependencies:   cctools, pkgconfig
Library Dependencies: ilmbase, zlib
Platforms:            darwin
License:              BSD
Maintainers:          Email: mcalhoun@macports.org, GitHub: MarcusCalhoun-Lopez
                      Policy: openmaintainer
$ port deps opencv3
Full Name: opencv3 @3.4.16_1
Build Dependencies:   cmake, pkgconfig
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg, tiff, webp, ilmbase, openexr
$ port deps openexr
Full Name: openexr @2.3.0_2
Build Dependencies:   cctools, pkgconfig
Library Dependencies: ilmbase, zlib
$ 

For really inquisitive, I'm attaching output of port rdeps opencv3. There's no mentioning of Clang either.

[mouse07410 (Mouse)]

Output of "port rdeps opencv3"

[mascguy (Christopher Nielsen)]

Can you provide more info... specifically regarding what type of Mac you're running on, the macOS version, etc?

[jmroot (Joshua Root)]

Replying to mascguy:

Can you provide more info... specifically regarding what type of Mac you're running on, the macOS version, etc?

And any variants used. (This is why we're incessantly asking for logs; the information in them saves a lot of questions.)

[mouse07410 (Mouse)]

Understood. iMac (Retina 5K, 27-inch, 2020), 3.6 GHz 10-Core Intel Core i9, MacOS 11.6.2.

$ port installed opencv3
The following ports are currently installed:
  opencv3 @3.4.16_0+avx2+contrib+java+nonfree+opencl+openmp+qt5+vtk
  opencv3 @3.4.16_1+avx2+contrib+java+nonfree+opencl+openmp+qt5+vtk (active)
$

[jmroot (Joshua Root)]

The dependencies seem to be reported correctly here.

% port deps opencv3 +avx2+contrib+java+nonfree+opencl+openmp+qt5+vtk
Full Name: opencv3 @3.4.16_1+avx2+contrib+java+nonfree+opencl+openmp+qt5+vtk
Build Dependencies:   cmake, pkgconfig, apache-ant, clang-9.0
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg,
                      tiff, webp, ilmbase, openexr, vtk, gflags, google-glog,
                      ceres-solver, libomp, qt5-qtbase, openjdk11

[mascguy (Christopher Nielsen)]

Replying to jmroot:

The dependencies seem to be reported correctly here.

% port deps opencv3 +avx2+contrib+java+nonfree+opencl+openmp+qt5+vtk
Full Name: opencv3 @3.4.16_1+avx2+contrib+java+nonfree+opencl+openmp+qt5+vtk
Build Dependencies:   cmake, pkgconfig, apache-ant, clang-9.0
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg,
                      tiff, webp, ilmbase, openexr, vtk, gflags, google-glog,
                      ceres-solver, libomp, qt5-qtbase, openjdk11

Yep! While I haven't been able to dig into this further - my day job commitments have been extra-intensive this week - my guess is that variant openmp is driving the dependency on Clang 9.

And that ​seems both reasonable, and correct. Though I'll defer to Josh and others, to confirm.

[mouse07410 (Mouse)]

my guess is that variant openmp is driving the dependency on Clang 9. And that ​seems both reasonable, and correct.

And the reason that dependency does not show on my machines?

$ port installed opencv3
The following ports are currently installed:
  opencv3 @3.4.16_1+avx2+contrib+java+nonfree+opencl+openmp+qt5+vtk (active)
$ port deps opencv3
Full Name: opencv3 @3.4.16_1
Build Dependencies:   cmake, pkgconfig
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg, tiff, webp, ilmbase, openexr
$

Also, I see that your output shows a ton more of dependencies than mine...?

Though I'll defer to Josh and others, to confirm.

Sure.

Also, if it's a legitimate dependency - how about either upgrading it to, e.g., clang-13, or making it accept whatever Macports clang is installed if it's 9.0 or newer?

My objection was not that it was using Macports clang (though there doesn't seem much sense in doing that, having Xcode clang around) - but that it (a) did not document it (at least on my system), and (b) downloaded an old version of clang.

[reneeotten (Renee Otten)]

Replying to mouse07410:

And the reason that dependency does not show on my machines?

$ port installed opencv3
The following ports are currently installed:
  opencv3 @3.4.16_1+avx2+contrib+java+nonfree+opencl+openmp+qt5+vtk (active)
$ port deps opencv3
Full Name: opencv3 @3.4.16_1
Build Dependencies:   cmake, pkgconfig
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg, tiff, webp, ilmbase, openexr
$

that's because you've installed opencv3 with non-default variants but are asking only for the dependencies of the port with its default variants. You have to specify the same variants with port deps as you've used with port install to get an accurate list of the dependencies as Joshua showed.

My objection was not that it was using Macports clang (though there doesn't seem much sense in doing that, having Xcode clang around) - but that it (a) did not document it (at least on my system), and (b) downloaded an old version of clang.

Xcode clang versions do not support openmp, if you remove that variant from the opencv3 port it will likely not install a MacPorts' version of clang anymore.

[jmroot (Joshua Root)]

Replying to mouse07410:

Also, if it's a legitimate dependency - how about either upgrading it to, e.g., clang-13, or making it accept whatever Macports clang is installed if it's 9.0 or newer?

The opencv3 port doesn't make the decision of which specific compiler to use, it just asks for C++11 and openmp 4.0 support and base chooses a compiler that meets those requirements.

[mouse07410 (Mouse)]

that's because you've installed opencv3 with non-default variants but are asking only for the dependencies of the port with its default variants

Excellent point! I did not know that :-(. Thanks!

The opencv3 port doesn't make the decision of which specific compiler to use, it just asks for C++11 and openmp 4.0 support and base chooses a compiler that meets those requirements.

Understood. But in that case, why didn't it use already-installed Macports clang-11 or (better yet) clang-13?

[mouse07410 (Mouse)]

The opencv3 port doesn't make the decision of which specific compiler to use, it just asks for C++11 and openmp 4.0 support and base chooses a compiler that meets those requirements.

Then, it's a problem with the base - which ignored already-installed clang-13 and clang-11?

[mascguy (Christopher Nielsen)]

FYI, @jmroot has updated MacPorts' overall Clang selection logic, to prefer newer MacPorts-provided versions:

​https://github.com/macports/macports-ports/commit/cc87e6d82b6c3fc6ae3e40f121204cedbcdefb88

After after syncing the ports tree, you'll see the change for the situation we're discussing here. (With the caveat that I just ran this test on macOS 10.13; results for your Big Sur setup might be different.)

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.13.6
BuildVersion:	17G14019

$ port deps opencv3 +openmp
Full Name: opencv3 @3.4.16_1+openmp
Build Dependencies:   cmake, pkgconfig, clang-12
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg, tiff, webp, ilmbase, openexr, libomp

[mouse07410 (Mouse)]

Thank you!

But something still doesn't appear right:

$ sw_vers
ProductName:	macOS
ProductVersion:	11.6.2
BuildVersion:	20G314
$ port deps opencv3 +openmp
Full Name: opencv3 @3.4.16_1+openmp
Build Dependencies:   cmake, pkgconfig, clang-12
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg, tiff, webp, ilmbase, openexr,
                      libomp
$ port installed | grep clang
  clang-11 @11.1.0_3+analyzer+libstdcxx (active)
  clang-13 @13.0.0_0+analyzer+libstdcxx (active)
  clang_select @2.2_0 (active)
  fftw-3 @3.3.10_0+clang11+gfortran+openmp (active)
$ 

Why, when clang-11 and clang-13 are installed, it chooses clang-12 that is not installed???

[jmroot (Joshua Root)]

Changing ports' dependencies based on what you happen to already have installed is inherently non-reproducible, so it's not something we want to start doing.

[mouse07410 (Mouse)]

FYI, @jmroot has updated MacPorts' overall Clang selection logic, to prefer newer MacPorts-provided versions

I understood this as "updated selection logic to pick the newest-acceptable version from what's installed, if none is available - install the newest acceptable"...?

Changing ports' dependencies based on what you happen to already have installed is inherently non-reproducible, so it's not something we want to start doing

I'm not sure I understand. What then is the "logic to prefer newer versions..."? And since clang-13 is "newer" than clang-12 - why was it not "preferred", especially since it was even installed?

[kencu (Ken)]

clang-13 was not chosen as the default clang compiler because it is too new, and is likely to cause too many builds to fail.

clang-12 was chosen.

You can manipulate this however you want on your own system by monkeying around with your default_compilers setting in macports.conf.

Just don't open any tickets if you do that :>

[kencu (Ken)]

as mentioned by Josh, what you have installed already cannot be used as a criteria for the compiler that is chosen.

I know it seems perhaps logical, and in the past I have tried that idea, but it doesn't work out in the broad mix of 100,000 builds and so it is not going to be supported as a plan.

[mouse07410 (Mouse)]

clang-13 was not chosen as the default clang compiler because it is too new, and is likely to cause too many builds to fail

OK, understood. I might disagree - but I get your point.

You can manipulate this however you want on your own system by monkeying around with your default_compilers setting in macports.conf

Would you mind please give some references/pointers? The installed /opt/local/etc/macports/macports.conf does not contain default_compilers setting, nor is it mentioned in ​https://guide.macports.org/chunked/internals.configuration-files.html.

I'd like to at lease see how to do it, and what are the capabilities/options/choices if I go that route.

Just don't open any tickets if you do that :>

Now that is harsh! ;-)

as mentioned by Josh, what you have installed already cannot be used as a criteria for the compiler that is chosen. I know it seems perhaps logical, and in the past I have tried that idea, but it doesn't work out in the broad mix of 100,000 builds and so it is not going to be supported as a plan.

Indeed, what I suggested did sound logical to me - but I have no experience actually maintaining Macports for many users, so can't argue with your point.

[kencu (Ken)]

Replying to mouse07410:

Would you mind please give some references/pointers?

add a line something like this to macports.conf

default_compilers  macports-clang-13 macports-clang-12 macports-clang-11 macports-clang-10 macports-clang-9.0 macports-clang-8.0 macports-gcc-11 macports-gcc-10 macports-gcc-9

and add clang in there wherever you want to.

a (nearly) full list of the compiler options is here:

​https://trac.macports.org/wiki/UsingTheRightCompiler

[kencu (Ken)]

Replying to mouse07410:

Just don't open any tickets if you do that :>

Now that is harsh! ;-)

Well, when it comes to ticket opening, I think you have to be fair.

If you have done something non-standard on your system out of a personal choice, that makes it different than the buildbots, and then you get a build error that the buildbot does not get, IMHO it is not fair to open tickets for situations like that.

Everyone has enough to debug / do without chasing dragons tails...

[mouse07410 (Mouse)]

add a line something like this to macports.conf . . .

Great, thanks!

and add clang in there wherever you want to.

Could you please explain? That line already has a whole bunch of macports-clang-xx...? Or do you mean plain clang entry to mean the the Xcode clang (whatever version the current installed Xcode happens to be)?

Also: right now (the default/installed Macports config), port uses Xcode clang for everything except what requires capabilities like +openmp that need macports-clang-xx. If I add default_compilers . . . line - would it force port to build with the first entry-clang of that line, rather than reverting to macports-clang only when Xcode clang can't do it?

a (nearly) full list of the compiler options is here: ​​https://trac.macports.org/wiki/UsingTheRightCompiler

Yeah, I checked that URL - it doesn't even have macports-clang-13...

If you have done something non-standard on your system out of a personal choice, that makes it different than the buildbots, and then you get a build error that the buildbot does not get, IMHO it is not fair to open tickets for situations like that

Yeah, I get your point. Though, if, e.g., something fails to build with macports-clang-13 - wouldn't you want to know...?

[kencu (Ken)]

Replying to mouse07410:

Or do you mean plain clang entry to mean the the Xcode clang (whatever version the current installed Xcode happens to be)?

yes

Also: right now (the default/installed Macports config), port uses Xcode clang for everything except what requires capabilities like +openmp that need macports-clang-xx. If I add default_compilers . . . line - would it force port to build with the first entry-clang of that line, rather than reverting to macports-clang only when Xcode clang can't do it?

whatever you want. macports-clang-13 to build everything by default: put it first. xcode clang to build everything by default: put it first. You da boss. MacPorts will bend to your will, and do your bidding.

a (nearly) full list of the compiler options is here: ​​https://trac.macports.org/wiki/UsingTheRightCompiler

Yeah, I checked that URL - it doesn't even have macports-clang-13...

I noticed that too -- feel free to add it! MacPorts can always use some help, if you know what you're doing.

Though, if, e.g., something fails to build with macports-clang-13 - wouldn't you want to know...?

Use your discretion.

[mouse07410 (Mouse)]

whatever you want. macports-clang-13 to build everything by default: put it first. xcode clang to build everything by default: put it first.

To clarify: port will try entries in default_compilers line until it finds one that fits specific port's constraints - then it will try to build that port with that compiler?

I.e., if my line says default_compilers clang macports-clang-13 macports-gcc-11, it will try to fit Xcode clang first, if cannot - macports-clang-13, if that doesn't fit either - macports-gcc-11, and if that one fails too- port will refuse to build the given port, complaining that it cannot find a suitable compiler?

it doesn't even have macports-clang-13

feel free to add it! MacPorts can always use some help, if you know what you're doing

Well,that one fix doesn't seem too complicated to me - done. Thanks!

if something fails to build with macports-clang-13 - wouldn't you want to know...?

Use your discretion

Understand perfectly. No worries here.

Currently, my macports.conf only adds proxy settings to its default content. If I add default_compilers - I don't need to reload anything, or refresh whatever cache, or such - right? And to revert to the original behavior, I can just comment that line out, and maybe port clean ... the port that fails to build with that line present?

Thanks!

Update I added the following at the end of macports.conf file:

# Compilers that Macports will try to use to build ports from source
default_compilers clang macports-clang-13 macports-gcc-11 macports-clang-11

It reflects what I have already installed. Concern:

$ port deps opencv3 +openmp
Full Name: opencv3 @3.4.16_1+openmp
Build Dependencies:   cmake, pkgconfig
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg, tiff, webp, ilmbase, openexr
$

does not show Build dependency on clang at all - it seems to want macports-clang-12 exactly, and doesn't care what default I set? Note: when I add macports-clang-12 to that line, the output of port deps does not change (clang does not appear there).

Commenting the default_compilers line out restores correct reporting:

$ port deps opencv3 +openmp
Full Name: opencv3 @3.4.16_1+openmp
Build Dependencies:   cmake, pkgconfig, clang-12
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg, tiff, webp, ilmbase, openexr,
                      libomp
$

[kencu (Ken)]

set this:

default_compilers macports-clang-13 macports-clang-12 clang

and then see what happens

[mouse07410 (Mouse)]

set this:

default_compilers macports-clang-13 macports-clang-12 clang

When I did that, I got

$ port deps opencv3 +openmp
Full Name: opencv3 @3.4.16_1+openmp
Build Dependencies:   cmake, pkgconfig, clang-13
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg, tiff, webp, ilmbase, openexr,
                      libomp
$ 

which seems correct. But I'd prefer to use Xcode clang for everything that can be built with it, reserving Macports clang for things that have "special needs". I'm afraid the above default_compilers would compile all the Macports source ports with macports-clang-13...?

[kencu (Ken)]

Of course, you are the BOSS!

so do this then:

default_compilers clang macports-clang-13 macports-clang-12

Whatever you want, you got

[kencu (Ken)]

So -- let's move you from the "needs help" group to the "provides help" group.

I know you see how this works.

It is time for you to become a net provider of information and fixes rather than a net consumer of fixes and information.

Many people, including me, have invested heavily in you, and I think you are now at the point to be helping other people with their MacPorts issues!

Welcome to the Team !!!!!!!

[mouse07410 (Mouse)]

so do this then:

default_compilers clang macports-clang-13 macports-clang-12

Unfortunately, while the above would correctly reflect my preferences - it seems to break the compiler selection algorithm (which IMHO requires some work):

$ tail /opt/local/etc/macports/macports.conf

# Compilers that Macports will try to use to build ports from source
#default_compilers clang macports-clang-13 macports-gcc-11 macports-clang-11
#default_compilers macports-clang-13 macports-clang-12 clang
default_compilers clang macports-clang-13 macports-clang-12
$
$ port deps opencv3 +openmp
Full Name: opencv3 @3.4.16_1+openmp
Build Dependencies:   cmake, pkgconfig
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg, tiff, webp, ilmbase, openexr
$ 

It seems that when user's first preference is Xcode clang - selection algorithm fails, but when one of the Macport clang is the first - port selects the right compiler:

$ tail /opt/local/etc/macports/macports.conf

# Compilers that Macports will try to use to build ports from source
#default_compilers clang macports-clang-13 macports-gcc-11 macports-clang-11
default_compilers macports-clang-13 clang macports-clang-12
#default_compilers clang macports-clang-13 macports-clang-12
$ 
$ port deps opencv3 +openmp
Full Name: opencv3 @3.4.16_1+openmp
Build Dependencies:   cmake, pkgconfig, clang-13
Library Dependencies: ffmpeg, zlib, bzip2, libpng, libjpeg-turbo, openjpeg, tiff, webp, ilmbase, openexr,
                      libomp
$ 

So -- let's move you from the "needs help" group to the "provides help" group. . . . Welcome to the Team !!!!!!!

Thank you! Honored to accept.

Of course, there's a ton of stuff about Macports that I don't know (yet???), but I can probably help with small issues, relieving those more experienced to taking care of the harder ones.

[kencu (Ken)]

Replying to mouse07410:

so do this then:

default_compilers clang macports-clang-13 macports-clang-12

Unfortunately, while the above would correctly reflect my preferences - it seems to break the compiler selection algorithm

default_compilers is a bigger stick.

In each case, base selected your first default compiler.

the openmp selection mechanism is a bit more subtle. looks like default_compilers overrides it.

To get what you want, which seems to be that you want 99.99% of the standard compiler selection mechanism working as it is, but you just want the order of macports-clang-N selection changing, you might be happier if you adjusted that list on your system slightly. It lives here:

​https://github.com/macports/macports-ports/blob/f26cfe7b3e3961d120d8b6212c219b9a752cddeb/_resources/port1.0/compilers/clang_compilers.tcl#L16

and just add your macports-clang-13 to the top of your list instead of lower down (as it is now)

(which IMHO requires some work).

I find it works exactly the way I would expect it to.

It's all transparent, and it's all just code. Read portconfigure.tcl and it is all sitting there.

If you have some reproducible examples of how it does not work, I'm sure everyone would like to see them.

[kencu (Ken)]

Replying to mouse07410:

So -- let's move you from the "needs help" group to the "provides help" group. . . . Welcome to the Team !!!!!!!

Thank you! Honored to accept.

Just to be clear here -- I have no say in who is a MacPorts committer.

I'm just talking about the "Team" of people who answer questions on these trac tickets rather than present questions for others to sort out all the time.

[mouse07410 (Mouse)]

In each case, base selected your first default compiler. . . . I find it works exactly the way I would expect it to . . .

I thought it should select the first compiler that satisfies all of the port constraints! Otherwise, there's no reason to give a list/set of compilers.

So, could you please explain the logic of default_compilers compiler selection? As I said above, I expected it to work like this:

• take the first entry and check it against the port requirements;
• if it matches - use it, otherwise proceed to the next entry in the list.
• if list exhausted and no acceptable compiler found - refuse to build the port.

This ticket seems to show that in reality it does not work that way.

Read portconfigure.tcl and it is all sitting there

I will, thanks.

If you have some reproducible examples of how it does not work, I'm sure everyone would like to see them

Does not this ticket qualify as a reproducible example, using port opencv3?

When clang is listed first in the default_compilers, followed by macports-clang-13, I would expect the "selector" to:

• pick clang - and find that it does not satisfy the +openmp port requirement;
• proceed to the next entry macports-clang-13, find that it satisfies all the port requirements;
• select macports-clang-13 to build opencv3 and display it as a build dependency.

In fact, only if macports-clang-13 is the first in the default_compilers list, it's picked. I think it's a reproducible evidence of the failure of the algorithm logic.

Or am I missing something?

Just to be clear here -- I have no say in who is a MacPorts committer. I'm just talking about the "Team" of people who answer questions on these trac tickets rather than present questions for others to sort out all the time.

I think I'll find it challenging enough to answer questions at this point. Committing code is probably a bit above my grade yet.

[kencu (Ken)]

Replying to mouse07410:

In each case, base selected your first default compiler. . . . I find it works exactly the way I would expect it to . . .

I thought it should select the first compiler that satisfies all of the port constraints!

default_compilers is a bigger stick. It assumes you want to use that compiler as the default, and it returns your compiler list before things like openmp are considered:

​https://github.com/macports/macports-base/blob/ee6096ce3789742a7961b98bb87abf06b48d2d12/src/port1.0/portconfigure.tcl#L1247

Does not this ticket qualify as a reproducible example, using port opencv3?

No, I don't find it does.

1. you indicated that you felt clang-9.0 was a bit old as the first macports-clang-N compiler to be used, and so Josh changed that, with already quite some opposition expressed about that change based on the newer compiler possibly breaking builds that currently work fine.
2. you wanted the fix to be an even newer macports-clang compiler. See point 1.
3. I suggested two different ways you could change things to the way you want them to be on your system. Either adjust _resources/port1.0/compilers/clang_compilers.tcl#L16 as you wish, or set default_compilers. In your case, looks like the first option suits you better as you want MacPorts to override your choices based on settings in the Portfiles, but YMMV.

Everything is working 100% as expected to me.

In fact, only if macports-clang-13 is the first in the default_compilers list, it's picked. I think it's a reproducible evidence of the failure of the algorithm logic.

Or am I missing something?

Perhaps. I find it is reproducible evidence of the algorithm working.

default_compilers is picking the first default compiler you specified. It does not do a lot of further analysis, as it makes it's decisions and returns before things like openmp are pondered. I would have to experiment a bit to see if explicitly blacklisting a compiler would make it move down the list (I think that it would, but I don't have an afternoon today for this kind of thing to prove it).

So as above, you might prefer:

To get what you want, which seems to be that you want 99.99% of the standard compiler selection mechanism working as it is, but you just want the order of macports-clang-N selection changing, you might be happier if you adjusted that list on your system slightly. It lives here:

​​https://github.com/macports/macports-ports/blob/f26cfe7b3e3961d120d8b6212c219b9a752cddeb/_resources/port1.0/compilers/clang_compilers.tcl#L16

and just add your macports-clang-13 to the top of your list instead of lower down (as it is now), or any order of compilers you care to use.

[mouse07410 (Mouse)]

Could you please help me understand what I'm missing? I'm trying to figure the logic of default_compilers behavior. 1860-lines-worth TCL script did not help me - last time I touched TCL was around 2003, and it does not seem to deal with default_compilers anyway.

I thought that:

• default_compilers provides a list of compilers to try, in order of preference;
• Entries from this list are checked until a fit is found, or the list is exhausted;
• "Fit" above means the compiler from the list entry satisfies all of the port constraints.

Why when opencv3 port is evaluated, if macports-clang-13 is listed first, it is picked, but if it's listed second - it is not listed in the "deps"?

default_compilers is picking the first default compiler you specified

In that case, why is it named "compilers", and contains more than one entry? When the first default compiler does not satisfy the "openmp" constraint - why doesn't "it" (whatever "it" means in the context) proceed to the next entry???

[kencu (Ken)]

I think you best forget about default_compilers then, and instead modify

​​​https://github.com/macports/macports-ports/blob/f26cfe7b3e3961d120d8b6212c219b9a752cddeb/_resources/port1.0/compilers/clang_compilers.tcl#L16

If you so choose.

My apologies for mentioning default_compilers to you. If the tcl is not readable for you, you will be too frustrated to attempt to use it.

[mouse07410 (Mouse)]

My apologies for mentioning default_compilers to you.

No apology necessary - and it did seem like a good idea (actually, still does).

If the tcl is not readable for you, you will be too frustrated to attempt to use it.

Hopefully, one does not need to be a Tcl expert to correctly modify configuration file?

I would like to get to the bottom of how the list provided by default_compilers affects the compiler selection, especially when this list has more than one entry. I guess it is not documented, but somebody probably knows...?

I think you best forget about default_compilers then, and instead modify clang_compilers.tcl

I'm trying to avoid actually modifying the "guts" of the Macports distro, limiting myself to adjusting the configuration. Though I admit that modifying clang_compilers.tcl would be the simplest (short of actually figuring out default_compilers behavior).