cmake +qt5 +docs +python39 hangs during build

[mouse07410 (Mouse)]

MacOS Big Sur 11.6.1, Xcode-13.1. Macports-installed Python-3.9.7.

Upgrade of CMake hangs at this place:

$ port installed cmake
The following ports are currently installed:
  cmake @3.21.3_0+docs+python39+qt5 (active)
$ sudo port -vvv upgrade cmake
.  .  .  .  .
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-v3.21.4-f65cebf51a2cf3af2017fd9b03c685c77da00c74'
[ 96%] Built target cmake-gui
[100%] Built target CMakeLibTests

Not sure where to get more detailed log, but checking /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-v3.21.4-f65cebf51a2cf3af2017fd9b03c685c77da00c74/CMakeFiles/CMakeError.log, I see

.  .  .  .  .
Run Build Command(s):/opt/local/bin/gmake -f Makefile cmTC_c4be8/fast && gmake  -f CMakeFiles/cmTC_c4be8.dir/build.make CMakeFiles/cmTC_c4be8.dir/build
gmake[1]: Entering directory '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-v3.21.4-f65cebf51a2cf3af2017fd9b03c685c77da00c74/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_c4be8.dir/CheckSymbolExists.c.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang   -pipe -Os -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -arch x86_64  -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -mmacosx-version-min=11.0 -std=gnu11 -MD -MT CMakeFiles/cmTC_c4be8.dir/CheckSymbolExists.c.o -MF CMakeFiles/cmTC_c4be8.dir/CheckSymbolExists.c.o.d -o CMakeFiles/cmTC_c4be8.dir/CheckSymbolExists.c.o -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/CMakeFiles/CMakeTmp/CheckSymbolExists.c
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/CMakeFiles/CMakeTmp/CheckSymbolExists.c:8:19: error: use of undeclared identifier 'environ'
  return ((int*)(&environ))[argc];
                  ^
1 error generated.
gmake[1]: *** [CMakeFiles/cmTC_c4be8.dir/build.make:79: CMakeFiles/cmTC_c4be8.dir/CheckSymbolExists.c.o] Error 1
gmake[1]: Leaving directory '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-v3.21.4-f65cebf51a2cf3af2017fd9b03c685c77da00c74/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:127: cmTC_c4be8/fast] Error 2

File /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/CMakeFiles/CMakeTmp/CheckSymbolExists.c:
/* */
#include <stdlib.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef environ
  return ((int*)(&environ))[argc];
#else
  (void)argc;
  return 0;
#endif
}
Checking whether CXX compiler has environ in stdlib.h failed to compile with the following output:
Change Dir: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/Source/kwsys/CMakeFiles/CMakeTmp
Run Build Command(s):/opt/local/bin/gmake -f Makefile cmTC_6162b/fast && gmake  -f CMakeFiles/cmTC_6162b.dir/buil
d.make CMakeFiles/cmTC_6162b.dir/build
gmake[1]: Entering directory '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_re
lease_tarballs_ports_devel_cmake/cmake/work/cmake-v3.21.4-f65cebf51a2cf3af2017fd9b03c685c77da00c74/Source/kwsys/C
MakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_6162b.dir/kwsysPlatformTestsCXX.cxx.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -DTEST_KWSYS_CXX_H
AS_ENVIRON_IN_STDLIB_H  -pipe -Os -stdlib=libc++ -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/Ma
cOSX.platform/Developer/SDKs/MacOSX.sdk -arch x86_64  -isysroot /Applications/Xcode.app/Contents/Developer/Platfo
rms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -mmacosx-version-min=11.0 -std=gnu++17 -MD -MT CMakeFiles/cmTC_6162
b.dir/kwsysPlatformTestsCXX.cxx.o -MF CMakeFiles/cmTC_6162b.dir/kwsysPlatformTestsCXX.cxx.o.d -o CMakeFiles/cmTC_
6162b.dir/kwsysPlatformTestsCXX.cxx.o -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macp
orts.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/Source/kwsys/kwsysPlatformTestsCXX.cxx
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/Source/kwsys/kwsysPlatformTestsCXX.cxx:53:13: error: use of undeclared identifier 'environ'
  char* e = environ[0];
            ^
1 error generated.
gmake[1]: *** [CMakeFiles/cmTC_6162b.dir/build.make:79: CMakeFiles/cmTC_6162b.dir/kwsysPlatformTestsCXX.cxx.o] Error 1
gmake[1]: Leaving directory '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-v3.21.4-f65cebf51a2cf3af2017fd9b03c685c77da00c74/Source/kwsys/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:127: cmTC_6162b/fast] Error 2
.  .  .  .  .
Run Build Command(s):/opt/local/bin/gmake -f Makefile cmTC_1d0fe/fast && gmake  -f CMakeFiles/cmTC_1d0fe.dir/buil
d.make CMakeFiles/cmTC_1d0fe.dir/build
gmake[1]: Entering directory '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_re
lease_tarballs_ports_devel_cmake/cmake/work/cmake-v3.21.4-f65cebf51a2cf3af2017fd9b03c685c77da00c74/Source/kwsys/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_1d0fe.dir/kwsysPlatformTestsCXX.cxx.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -DTEST_KWSYS_CXX_STAT_HAS_ST_MTIM  -pipe -Os -stdlib=libc++ -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -arch x86_64  -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -mmacosx-version-min=11.0 -std=gnu++17 -MD -MT CMakeFiles/cmTC_1d0fe.dir/kwsysPlatformTestsCXX.cxx.o -MF CMakeFiles/cmTC_1d0fe.dir/kwsysPlatformTestsCXX.cxx.o.d -o CMakeFiles/cmTC_1d0fe.dir/kwsysPlatformTestsCXX.cxx.o -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/Source/kwsys/kwsysPlatformTestsCXX.cxx
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/Source/kwsys/kwsysPlatformTestsCXX.cxx:12:15: error: no member named 'st_mtim' in 'stat'
  (void)stat1.st_mtim.tv_sec;
        ~~~~~ ^
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/Source/kwsys/kwsysPlatformTestsCXX.cxx:13:15: error: no member named 'st_mtim' in 'stat'
  (void)stat1.st_mtim.tv_nsec;
        ~~~~~ ^
2 errors generated.
gmake[1]: *** [CMakeFiles/cmTC_1d0fe.dir/build.make:79: CMakeFiles/cmTC_1d0fe.dir/kwsysPlatformTestsCXX.cxx.o] Error 1
gmake[1]: Leaving directory '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-v3.21.4-f65cebf51a2cf3af2017fd9b03c685c77da00c74/Source/kwsys/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:127: cmTC_1d0fe/fast] Error 2

Seems some silly forever-loop, with Python taking 98% of CPU.

[mouse07410 (Mouse)]

Snapshot of what the "hanging" Python seems to be doing

[mouse07410 (Mouse)]

Probably irrelevant, but I don't know what else to provide

[mouse07410 (Mouse)]

Screen log of the complete build, up to the point where it hangs

[ned-deily (Ned Deily)]

FWIW, the cmake source upgrade from cmake @3.21.3_0+docs+python39 to @3.21.4_0 hangs in Python for me as well, that is, without +qt5. This is also on 11.6.1 with Xcode 13.1.

[sambthompson (Sam Thompson)]

Replying to ned-deily:

FWIW, the cmake source upgrade from cmake @3.21.3_0+docs+python39 to @3.21.4_0 hangs in Python for me as well, that is, without +qt5. This is also on 11.6.1 with Xcode 13.1.

Seeing the same symptoms attempting to upgrade from cmake @3.21.3_0+docs+python39 to @3.21.4_0, on 10.11.6 with Xcode 8.2.1.

[adsche]

This happened to me too. The hanging process is apparently building the Sphinx documentation:

$ sudo pgrep -lf Python
93916 /opt/local/Library/Frameworks/Python.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python /opt/local/bin/sphinx-build-3.9 -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/Utilities/Sphinx -d /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/Utilities/Sphinx/doctrees -b html /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/Help /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.21.4/Utilities/Sphinx/html

But no output seems to be generated in any of the folders on the command line. Therefore I wanted to try the +python39-docs variant but it did still try to install +docs (I did clean in between).

port install cmake -docs worked, but +python39 seems to imply +docs, unfortunately.

I am not sure whether it is actually hanging or just taking really long, but I did not see any output generated in the output directory for 30 minutes.

EDIT ok, actually cmake only needs python for generating the docs so the dependency makes sense.

[mouse07410 (Mouse)]

Rest assured that it is hanging, rather than merely "taking a long time", because before reporting this I left the update running overnight - and in the morning the progress indicator remained at the same place.

[mmpestorich (Mike M Pestorich)]

Also seeing the same issue on macOS 12.0.1 and Xcode 13.1. Only way to get cmake to build was with port install make -docs -python39.

[pdvnl]

for one of my macs running 10.13 high sierra (the others worked fine) i have the same problem: after

[100%] Built target CMakeLibTests

there is no more visible progress, but activity monitor shows around 100% cpu for python.

pstree 97542
-+= 97542 macports /opt/local/libexec/macports/bin/tclsh8.5 /opt/local/bin/port -dv upgrade cmake -docs-python38+python39
 \-+- 97549 macports sh -c  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_relea
   \-+- 97550 macports /Library/Developer/CommandLineTools/usr/bin/make -j4 -w all VERBOSE=ON
     \-+- 97553 macports /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/Makefile2 all
       \-+- 97581 macports /Library/Developer/CommandLineTools/usr/bin/make -f Utilities/Sphinx/CMakeFiles/documentation.dir/bu
         \-+- 97583 macports /bin/sh -c cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_mac
           \--- 97584 macports /opt/local/Library/Frameworks/Python.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/

[mascguy (Christopher Nielsen)]

@michaelld, if this isn't easily fixable, there's also the option of expediting the switch to subports for the GUI and Docs variants. (If you're on-board with that change?) See issue:63830

The work isn't complete just yet - had to temporarily table it, in favor of assisting with the OpenSSL 3 migration - but we're close. The proposed [not-quite-working] changes are visible now, via cmake-devel.

[michaelld (Michael Dickens)]

In my testing, this is a Sphinx issue. The actual build hangs with this command:

/opt/local/bin/sphinx-build-3.9 -c /opt/local/var/macports/build/_opt_sources_MacPorts_ports_github_macports_devel_cmake/cmake/work/cmake-3.21.4/Utilities/Sphinx -d /opt/local/var/macports/build/_opt_sources_MacPorts_ports_github_macports_devel_cmake/cmake/work/cmake-3.21.4/Utilities/Sphinx/doctrees -b html /opt/local/var/macports/build/_opt_sources_MacPorts_ports_github_macports_devel_cmake/cmake/work/cmake-3.21.4/Help /opt/local/var/macports/build/_opt_sources_MacPorts_ports_github_macports_devel_cmake/cmake/work/cmake-3.21.4/Utilities/Sphinx/html > build-html.log

I tried Sphinx from Py37 and Py38 & they hang as well. The output file build-html.log is the same regardless of which is used.

[michaelld (Michael Dickens)]

FYI this testing is on a Mac Mini M1 running macOS 11.6.1 20G224 ; Xcode 13.1 13A1030d

[michaelld (Michael Dickens)]

I will try reverting Sphinx to the prior version & see if that resolves this issue

[michaelld (Michael Dickens)]

Nope not Sphinx ... reverting those back a few releases doesn't help

[michaelld (Michael Dickens)]

ah... progress: looks like py-docutils 0.18 is the culprit ... reverting to 0.17.1 resolves the issue!

[michaelld (Michael Dickens)]

In ebe02cae9dab3d68ac6f69c53d3df124b7922b84/macports-ports (master):

py-docutils: add commit r8555 to address various usage issues

Closes: #63765
Closes: #63739
Closes: #63727

[sambthompson (Sam Thompson)]

Replying to michaelld:

py-docutils: add commit r8555 to address various usage issues

Thanks, Michael; can confirm this fix to py-docutils allows cmake to upgrade OK.

[michaelld (Michael Dickens)]

@sambthompson : great! hopefully except for Py27 this fix will do the trick across both macOS & CPU & support Python