mongodb @4.2.3: error: constexpr function never produces a constant expression

[rseichter (Ralph Seichter)]

While the MongoDB 4.2.3 port built OK on macOS Catalina, build attempts on High Sierra failed for me. Please see attached log.

[rseichter (Ralph Seichter)]

Build log

[jmroot (Joshua Root)]

:debug:sysinfo Xcode none

What version of the CLTs and clang do you have installed?

[jmroot (Joshua Root)]

Oh never mind, a portgroup already told us.

:debug:main compiler clang 1000.10.44.4 not blacklisted because it doesn't match {clang <= 318.0.61}

That seems too new for Xcode 9 (the buildbot has Xcode 9.4.1 with clang 902.0.39.2) but too old for Xcode 10, which according to the wiki shipped with clang-1000.11.45.2.

[kencu (Ken)]

Same failure for me as well, on my High Sierra system. My system seems fully up to date, and lists no updates available. Opening Xcode shows no instructions to update anything. It passes the configure Xcode 10 test, but does not have an Xcode that enables that __builtin function it appears.

However, interestingly, it does seem to have slightly different versions of Xcode installed in the Xcode installation and in the Command Line Tools installation...

$     echo "macOS $(sw_vers -productVersion) $(sw_vers -buildVersion)"; echo "Xcode $(xcodebuild -version | awk -v ORS=' ' '{print $NF}')"
macOS 10.13.6 17G11023
Xcode 10.1 10B61 

$ /usr/bin/clang -v
Apple LLVM version 10.0.0 (clang-1000.11.45.5)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

$ /Library/Developer/CommandLineTools/usr/bin/clang -v
Apple LLVM version 10.0.0 (clang-1000.10.44.4)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Forcing the mongodb build against macports-clang-9.0 works.

[ryandesign (Ryan Carsten Schmidt)]

It built fine for me on High Sierra with Xcode 9.

I'm not going to have time for awhile to look into why it failed for you on High Sierra with Xcode 10. You should probably report the problem to the developers of mongodb so that they can fix it for the next version.

[rseichter (Ralph Seichter)]

I opened ​https://jira.mongodb.org/browse/SERVER-45922 but I don't have high hopes. The last MongoDB ticket I filed is now open for almost 1.5 years.

[kencu (Ken)]

You got the quick workaround, I hope:

sudo port -v install mongodb configure.compiler=macports-clang-9.0

will get it in for you.

[rseichter (Ralph Seichter)]

Build log #2

[rseichter (Ralph Seichter)]

Ken: I tried that command, but was not able to build MongoDB (see attachment "main2.log").

[kencu (Ken)]

I thought it would automatically add it for you, but please check to see if you have clang-9.0 installed already:

sudo port -v installed | grep clang-9.0 | grep active

if by some chance you don't, then install clang-9.0

sudo port -v install clang-9.0

and then please install mongo again

sudo port -v install mongodb configure.compiler=macports-clang-9.0

if that all fails -- I'm uncertain why. It works for me.

[rseichter (Ralph Seichter)]

Build log #3

[rseichter (Ralph Seichter)]

I am sorry, but it still does not work (see main3.log). If I understand this correctly, there's a complaint about sasl2 files missing, but cyrus-sasl2 is installed:

→ port installed | grep sasl
  cyrus-sasl2 @2.1.27_2+kerberos (active)

[ryandesign (Ryan Carsten Schmidt)]

attachment:main2.log​ shows two build attempts: the first using /usr/bin/clang, the second encountering circular dependencies.

attachment:main3.log​ shows four build attempts: the first two are the same as in main2.log; the third and fourth show this error:

:info:build Couldn't find SASL header/libraries
:info:build See /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_databases_mongodb/mongodb/work/mongodb-src-r4.2.3/build/scons/config.log for details

If you attach that config.log file, we might learn more about why that happened.

[rseichter (Ralph Seichter)]

Thanks for your patience. I have attached config.log. What may interest you also:

→ find /opt/local -type f -name sasl.h
/opt/local/include/sasl/sasl.h

[kencu (Ken)]

looks like our old friend once again:

ld: unexpected token: !tapi-tbd-v3 file '/System/Library/Frameworks//Security.framework/Security.tbd' for architecture x86_64

which will hopefully soon be history once the PR for the new ld64-latest is committed shortly.

for now, do this please:

sudo port -f uninstall ld64
sudo port -v install ld64

and then clean and try your mongodb reinstall like this:

sudo port clean mongodb
sudo port -v install mongodb configure.compiler=macports-clang-9.0

[rseichter (Ralph Seichter)]

Replying to kencu:

looks like our old friend once again [...]

I followed the steps you have kindly provided, alas the build still fails, appartently still due to SASL problems (see main4.log).

[kencu (Ken)]

please show me the output of :

port -v installed | grep ld64

and

which ld
ld -v

thank you.

[kencu (Ken)]

For future reference, the main.log is helpful but is not showing the error that leads to the sasl libraries not being found. Those errors are in the config.log, so we'll need each log every time to help you make progress.

Always clean between build attempts with sudo port clean mongodb. It's unfortunate that has to be a manual step all the time, but it is.

So far, the error seems to be with your linker, ergo my requests above.

[rseichter (Ralph Seichter)]

Here we go:

→ port -v installed | grep ld64
  ld64 @3_1 (active) platform='darwin 17' archs='x86_64' date='2020-02-03T23:57:02+0100'
  ld64-latest @274.2_2+llvm90 (active) platform='darwin 17' archs='x86_64' date='2020-02-03T04:21:40+0100'
→ which ld
/opt/local/bin/ld
→ ld -v
@(#)PROGRAM:ld  PROJECT:ld64-274.2
configured to support archs: i386 x86_64 x86_64h armv6 armv7 armv7s armv7m armv7k arm64 (tvOS)
LTO support using: LLVM version 9.0.1

[kencu (Ken)]

yes, wrong linker. The instructions I gave you above should have installed the right one. I don't know why they did not. Please do this to force the right linker:

sudo port -f uninstall ld64
sudo port -v install ld64 +ld64_xcode

and then clean and retry your mongodb installation once again. I'm predicting success!

If you get failure, recheck your linker. you should see ld64 +ld64_xcode installed

[kencu (Ken)]

BTW, I've been working all this past week at fixing up this very problem by upgrading ld64-latest to something more current that can speak TAPI and handle those TBD linker files.

[rseichter (Ralph Seichter)]

Replying to kencu:

I'm predicting success!

You predicted correctly. ;-) Thank you once again for your help and patience. That old MacPro of mine is incapable of running Catalina, but now it can run the latest MongoDB version on High Sierra.

[acmorrow (Andrew C. Morrow)]

Hi, I'm one of the maintainers of the MongoDB build system. We saw that you opened ​https://jira.mongodb.org/browse/SERVER-45922, but having come over to this issue it appears that you have it resolved. The details of what was fixed seem very MacPorts specific, and it isn't an environment I know well, so the details of the fix are a bit opaque to me. It looks like the fix was to require the Xcode linker? If so, that would make sense to me, as Xcode is the only toolchain we use in our macOS CI build and test. Still, I wanted to check in to see whether there was anything we at MongoDB could do in our build system that would have avoided this issue for you, or at least made it simpler to diagnose?

[kencu (Ken)]

The linker error was very MacPorts-specific, and should have a permanent fix shortly.

Whatever is causing XCode's clang to not see the __builtin function which it is supposed to support (as per the initial log submitted, and see 60032#comment:4 above) is not as clear at present. We are working around that manually by forcing a build with the open source clang-9.0 from the llvm project.

[ryandesign (Ryan Carsten Schmidt)]

The fact that we need to update the MacPorts copy of the Apple linker is #57587.

But I don't see what all this talk of a linker has to do with the errors shown in the first main.log​ attached here:

:info:build In file included from src/mongo/s/request_types/set_shard_version_request.cpp:32:
:info:build In file included from src/mongo/s/request_types/set_shard_version_request.h:35:
:info:build In file included from src/mongo/client/connection_string.h:40:
:info:build In file included from src/mongo/platform/mutex.h:38:
:info:build In file included from src/mongo/util/duration.h:38:
:info:build src/mongo/platform/overflow_arithmetic.h:97:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoSignedMultiplyOverflow64(long lhs, long rhs, long* product) {
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:98:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_mul_overflow(lhs, rhs, product);
:info:build            ^
:info:build src/mongo/platform/overflow_arithmetic.h:101:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoSignedMultiplyOverflow64(long long lhs, long long rhs, long long* product) {
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:102:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_mul_overflow(lhs, rhs, product);
:info:build            ^
:info:build src/mongo/platform/overflow_arithmetic.h:105:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoUnsignedMultiplyOverflow64(unsigned long lhs,
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:108:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_mul_overflow(lhs, rhs, product);
:info:build            ^
:info:build src/mongo/platform/overflow_arithmetic.h:111:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoUnsignedMultiplyOverflow64(unsigned long long lhs,
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:114:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_mul_overflow(lhs, rhs, product);
:info:build            ^
:info:build src/mongo/platform/overflow_arithmetic.h:117:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoSignedAddOverflow64(long lhs, long rhs, long* sum) {
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:118:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_add_overflow(lhs, rhs, sum);
:info:build            ^
:info:build src/mongo/platform/overflow_arithmetic.h:121:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoSignedAddOverflow64(long long lhs, long long rhs, long long* sum) {
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:122:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_add_overflow(lhs, rhs, sum);
:info:build            ^
:info:build src/mongo/platform/overflow_arithmetic.h:125:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoUnsignedAddOverflow64(unsigned long lhs,
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:128:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_add_overflow(lhs, rhs, sum);
:info:build            ^
:info:build src/mongo/platform/overflow_arithmetic.h:131:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoUnsignedAddOverflow64(unsigned long long lhs,
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:134:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_add_overflow(lhs, rhs, sum);
:info:build            ^
:info:build src/mongo/platform/overflow_arithmetic.h:137:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoSignedSubtractOverflow64(long lhs, long rhs, long* difference) {
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:138:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_sub_overflow(lhs, rhs, difference);
:info:build            ^
:info:build src/mongo/platform/overflow_arithmetic.h:141:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoSignedSubtractOverflow64(long long lhs, long long rhs, long long* difference) {
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:142:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_sub_overflow(lhs, rhs, difference);
:info:build            ^
:info:build src/mongo/platform/overflow_arithmetic.h:145:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoUnsignedSubtractOverflow64(unsigned long lhs,
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:148:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_sub_overflow(lhs, rhs, sum);
:info:build            ^
:info:build src/mongo/platform/overflow_arithmetic.h:151:16: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
:info:build constexpr bool mongoUnsignedSubtractOverflow64(unsigned long long lhs,
:info:build                ^
:info:build src/mongo/platform/overflow_arithmetic.h:154:12: note: subexpression not valid in a constant expression
:info:build     return __builtin_sub_overflow(lhs, rhs, sum);
:info:build            ^

The linker errors only occurred due to Ken's suggestion to use a newer MacPorts clang, which IMHO should not be necessary—unless it is felt that these compiler error messages are in fact a bug in the Xcode 10 version of the compiler which the developers of mongodb do not wish to work around, in which case sure we can blacklist the affected Xcode compilers.

[ryandesign (Ryan Carsten Schmidt)]

And I didn't mean to close this.

[kencu (Ken)]

Replying to ryandesign:

The linker errors only occurred due to Ken's suggestion to use a newer MacPorts clang

Do you think? I think his Xcode clang would have called in the outdated /opt/local/bin/ld as well, eventually, and found it broken.

Actually, that's something I would like check -- up to now, I thought the Xcode clang used the first ld in the path, but I've not actually tried to prove that. If it ignores /opt/local/bin/ld and uses some baked-in ld that would be news to me.

Anyway, I'm just about to hit the "commit" button on that ld64 unless someone pipes up shortly, and this *.tbd linker error SHOULD be history.

[ryandesign (Ryan Carsten Schmidt)]

...which still doesn't explain why the constexpr errors occurred or what the solution to those is, other than using a newer compiler.

[ryandesign (Ryan Carsten Schmidt)]

...bearing in mind again that the older compiler in Xcode 9 was happy to compile this code.

[ryandesign (Ryan Carsten Schmidt)]

...bearing in mind that I forgot we are blacklisting clang < 1000, so actually with Xcode 9, MacPorts clang 9 would be used instead.

Upstream can now reproduce the problem with Xcode 10.0 and 10.1, but not with 10.2. They might not fix that, and might instead require a minimum of Xcode 10.2.

[ryandesign (Ryan Carsten Schmidt)]

In 413f149911924d7442c4ebe5c0923eabb7555005/macports-ports (master):

mongodb: Blacklist clang from Xcode 10.0 and 10.1

Closes: #60032