qt5-qtwebengine @5.15.2 build error on Monterey

[amake (Aaron Madlon-Kay)]

After using the ​`use_xcode yes` workaround to get qt5-qtbase installed, I find that qt5-qtwebengine fails with the following:

:info:build TOOL_VERSION=1604712156 ../../3rdparty/chromium/build/toolchain/mac/linker_driver.py -Wcrl,strippath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip ../../../../../../../../../../../../Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -B /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/  -Wcrl,dsym,. -Wcrl,dsymutilpath,../../../../../../../../../../../../Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/dsymutil  -Wl,-fatal_warnings -stdlib=libc++ -arch x86_64 -Wl,-no_data_in_code_info -Wl,-no_function_starts -isysroot ../../../../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -mmacosx-version-min=12.0 -Wl,-ObjC -Wcrl,strip,-x,-S -o "./transport_security_state_generator" -Wl,-filelist,"./transport_security_state_generator.rsp"   -framework ApplicationServices -framework AppKit -lbsm -framework CoreFoundation -framework IOKit -framework OpenDirectory -lpmenergy -lpmsample -framework Security
:info:build Undefined symbols for architecture x86_64:
:info:build   "std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator<unsigned short> >::shrink_to_fit()", referenced from:
:info:build       base::UTF8ToUTF16(char const*, unsigned long, std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator<unsigned short> >*) in libbase.a(base_jumbo_20.o)
:info:build       base::WideToUTF16(wchar_t const*, unsigned long, std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator<unsigned short> >*) in libbase.a(base_jumbo_20.o)
:info:build ld: symbol(s) not found for architecture x86_64
:info:build clang: error: linker command failed with exit code 1 (use -v to see invocation)
:info:build Traceback (most recent call last):
:info:build   File "../../3rdparty/chromium/build/toolchain/mac/linker_driver.py", line 287, in <module>
:info:build     Main(sys.argv)
:info:build   File "../../3rdparty/chromium/build/toolchain/mac/linker_driver.py", line 97, in Main
:info:build     subprocess.check_call(compiler_driver_args, env=env)
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 190, in check_call
:info:build     raise CalledProcessError(retcode, cmd)
:info:build subprocess.CalledProcessError: Command '['../../../../../../../../../../../../Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++', '-B', '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/', '-Wl,-fatal_warnings', '-stdlib=libc++', '-arch', 'x86_64', '-Wl,-no_data_in_code_info', '-Wl,-no_function_starts', '-isysroot', '../../../../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk', '-mmacosx-version-min=12.0', '-Wl,-ObjC', '-o', './transport_security_state_generator', '-Wl,-filelist,./transport_security_state_generator.rsp', '-framework', 'ApplicationServices', '-framework', 'AppKit', '-lbsm', '-framework', 'CoreFoundation', '-framework', 'IOKit', '-framework', 'OpenDirectory', '-lpmenergy', '-lpmsample', '-framework', 'Security']' returned non-zero exit status 1

[abey79 (Antoine Beyeler)]

I too have an error when installating qt5-qtwebengine on Monterey/M1, although the error is different:

:info:build Project MESSAGE: Running: /opt/local/var/macports/build/_Users_hhip_src_macports-ports_aqua_qt5/qt5-qtwebengine/work/qtwebengine-everywhere-src-5.15.2/src/3rdparty/gn/out/Release/gn gen /opt/local/var/macports/build/_Users_hhip_src_macports-ports_aqua_qt5/qt5-qtwebengine/work/qtwebengine-everywhere-src-5.15.2/src/core/release --script-executable=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2 --args='use_qt=true init_stack_vars=false is_component_build=false is_shared=true enable_debugallocation=false enable_media_remoting=false enable_message_center=false enable_nacl=false enable_remoting=false enable_reporting=false enable_resource_whitelist_generation=false enable_swiftshader=false angle_enable_swiftshader=false enable_web_auth=true enable_web_speech=false enable_widevine=true forbid_non_component_debug_builds=false has_native_accessibility=false safe_browsing_mode=0 toolkit_views=false treat_warnings_as_errors=false use_allocator_shim=false use_allocator="none" use_custom_libcxx=false closure_compile=false optimize_webui=false v8_use_external_startup_data=false strip_absolute_paths_from_debug_symbols=false use_jumbo_build=true jumbo_file_merge_limit=8 jumbo_build_excluded=["browser"] enable_precompiled_headers=false is_official_build=true is_debug=false symbol_level=0 blink_symbol_level=0 remove_v8base_debug_symbols=true optimize_for_size=true is_clang=true use_sysroot=false use_system_xcode=true clang_base_path="/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr" clang_use_chrome_plugins=false mac_deployment_target="12.0" mac_sdk_min="12.0" use_external_popup_menu=false angle_enable_vulkan=false enable_basic_printing=true enable_print_preview=true enable_pdf=true enable_plugins=true enable_spellcheck=true enable_webrtc=true proprietary_codecs=false enable_extensions=true use_kerberos=true have_nodejs=false use_browser_spellchecker=false qtwebengine_target="/opt/local/var/macports/build/_Users_hhip_src_macports-ports_aqua_qt5/qt5-qtwebengine/work/qtwebengine-everywhere-src-5.15.2/src/core/release:QtWebEngineCore"' --root=/opt/local/var/macports/build/_Users_hhip_src_macports-ports_aqua_qt5/qt5-qtwebengine/work/qtwebengine-everywhere-src-5.15.2/src/3rdparty/chromium 
:info:build ERROR at //third_party/zlib/BUILD.gn:100:9: Assertion failed.
:info:build         assert(false, "Unsupported ARM OS")
:info:build         ^-----
:info:build Unsupported ARM OS
:info:build See //ui/gfx/BUILD.gn:306:5: which caused the file to be included.
:info:build     "//third_party/zlib",
:info:build     ^-------------------
:info:build Project ERROR: GN run error!
:info:build make[2]: *** [sub-gn_run-pro-make_first] Error 3
:info:build make[2]: Leaving directory `/opt/local/var/macports/build/_Users_hhip_src_macports-ports_aqua_qt5/qt5-qtwebengine/work/qtwebengine-everywhere-src-5.15.2/src/core'
:info:build make[1]: *** [sub-core-make_first] Error 2
:info:build make[1]: Leaving directory `/opt/local/var/macports/build/_Users_hhip_src_macports-ports_aqua_qt5/qt5-qtwebengine/work/qtwebengine-everywhere-src-5.15.2/src'
:info:build make: *** [sub-src-make_first] Error 2
:info:build make: Leaving directory `/opt/local/var/macports/build/_Users_hhip_src_macports-ports_aqua_qt5/qt5-qtwebengine/work/qtwebengine-everywhere-src-5.15.2'
:info:build Command failed:  cd "/opt/local/var/macports/build/_Users_hhip_src_macports-ports_aqua_qt5/qt5-qtwebengine/work/qtwebengine-everywhere-src-5.15.2" && /usr/bin/make -j10 -w 
:info:build Exit code: 2
:error:build Failed to build qt5-qtwebengine: command execution failed
:debug:build Error code: CHILDSTATUS 61211 2
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$callback {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec -callback portprogress::target_progress_callback build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"

I wonder if ​https://github.com/macports/macports-ports/pull/12595 solves this issue.

[chrstphrchvz (Christopher Chavez)]

​PR 12595 does solve the issue in comment:3, but it is now blocked by the issue reported in this ticket.

[jmroot (Joshua Root)]

Given that this spends 4 hours building before it finally fails, can we please mark it as known_fail in the meantime?

[chrstphrchvz (Christopher Chavez)]

As I mentioned on ​https://github.com/macports/macports-ports/pull/12595#issuecomment-973304948:

I’ve not made any progress on the linker error. I might be wrong about it having sonething to do with system libc++: maybe it instead involves very recent Xcode clang, or a subtle bug in chromium code exposed by newer clang. One reason similar issues aren’t being widely reported could be that the relevant code basically no longer exists in newer chromium and qtwebengine: string16 was replaced by std::u16string. I would guess that change is not something trivial to backport.

At the moment I can only think of trying to separate out the relevant source file chromium/src/base/strings/utf_string_conversions.cc from the “jumbo” build, but I don’t expect the error to go away. Something else to try for investigating this more deeply would be to see if the issue also exists in some version of mini_chromium.

[chrstphrchvz (Christopher Chavez)]

Replying to jmroot:

Given that this spends 4 hours building before it finally fails, can we please mark it as known_fail in the meantime?

See ​https://github.com/macports/macports-ports/pull/13105

[chrstphrchvz (Christopher Chavez)]

In 6899a98cef46d1e22c54f25201059713facc2cab/macports-ports (master):

qt5-qtwebengine: does not build on arm64, macOS 12

[skip ci]

See: #62059
See: #63725

[chrstphrchvz (Christopher Chavez)]

I may have identified a workaround for this issue. Compiling src/3rdparty/chromium/base/strings/string16.cc separately (by adding it to jumbo_excluded_sources in src/3rdparty/chromium/base/BUILD.gn) outputs src/core/release/obj/base/base/string16.o, which appears to reliably contain base::string16::reserve(unsigned long). shrink_to_fit(), reserve(), and reserve(0) are equivalent in C++17 and earlier (various versions of libc++ have even implemented shrink_to_fit() as reserve() or vice versa in the include/c++/v1/string header file). Since string16.cc is currently being compiled with -std=c++14, I would try replacing dest_str->shrink_to_fit() with dest_str->reserve(0) in src/3rdparty/chromium/base/strings/utf_string_conversions.cc (I don’t know if it looks silly to do reserve(0) right after doing reserve(dest_len32) edit: misread resize(dest_len32), which would not be the same as reserve(dest_len32)).

I’m not sure what the right conditions are for getting string16.o to contain a symbol for base::string16::shrink_to_fit(). So far the only way I’ve managed to do so is to add a dummy function void Dummy(string16& str) { str->shrink_to_fit(); } to base::string16_internals in string16.cc (with corresponding declaration in string16.h), and compile string16.cc with the macOS 10.15 SDK. The clang version/vendor did not seem to matter (tried Xcode clang 13.2, MacPorts LLVM clang 9, and MacPorts LLVM clang 11); maybe that suggests this is more likely to be a libc++ bug in the macOS 12 SDKs. Edit: I don’t think I was careful with which version I used for different steps. I think it may make a difference since MacPorts’ clang seems to use its own libc++ headers by default.

[chrstphrchvz (Christopher Chavez)]

Someone reported a very similar issue to Qt: ​https://bugreports.qt.io/browse/QTBUG-98813

[cgilles (HumanDynamo)]

Qt5-qwebengine build fail too for me with the same error, for target x86_64 and MacOS SDK 10.13. MacOS host version : Big Sur 11.6.1, Xcode version : 13.1 (13A1030d)

[chrstphrchvz (Christopher Chavez)]

In bd858ac96b4181de1d388f8e355b852577264dcd/macports-ports (master):

qt5-qtwebengine: update to 5.15.7 (​https://github.com/macports/macports-ports/pull/12595)

[skip ci]

Newer QtWebEngine 5.15.x releases are tagged on git repository
Fetch archives from GitHub mirrors
Requires headers generated by syncqt.pl:
apply from patch to avoid build dependency on perl

Requires build dependency Node.js as of 5.15.3;
specify LTS version nodejs16 as fallback

Drop patch-qtwebengine_sdk.diff in favor of upstream approach

Rebase patch-qtwebengine_chromium_static_page_size.diff
(still needed for macOS 11 SDK even on x86_64)

macOS on ARM64 supported as of 5.15.3
Fixes: #62059

Workaround Undefined symbols error on macOS 12 Monterey
Fixes: #63725