python27 +universal fails to build universal extension modules in standard library

[rmottola (Riccardo)]

recently, ICU 65 was made to depend on python 2.7 so that it builds (and it did work(. On a different machine I also run upgrade of ICU, python 2.7 gets pulled in and is updated, but then build fails:

config.status: creating samples/Makefile
config.status: creating samples/date/Makefile
config.status: creating samples/cal/Makefile
config.status: creating samples/layout/Makefile
Not rebuilding data/rules.mk, assuming prebuilt data in data/in
Spawning Python to generate test/testdata/rules.mk...
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 163, in _run_module_as_main
    mod_name, _Error)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 111, in _get_module_details
    __import__(mod_name)  # Do not catch exceptions initializing package
  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_icu/icu/work/icu/source-i386/python/icutools/databuilder/__init__.py", line 4, in <module>
    from collections import namedtuple
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/collections.py", line 20, in <module>
    from _collections import deque, defaultdict
ImportError: No module named _collections
configure: error: Python failed to run; see above error.
./runConfigureICU: ./configure failed
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_icu/icu/work/icu/source-i386" && ./runConfigureICU MacOSX --prefix=/opt/local --disable-layoutex --disable-samples --enable-static 
Exit code: 1

is this still an ICU problem or actually an issue with python 2.7

[jmroot (Joshua Root)]

Seems to be python that's broken. First of all, if you start python2.7 interactively and run import collections do you get the same error? If so, the _collections module must have not been installed for some reason. That reason might be difficult to diagnose without access to a Leopard system, but the log from building python27 might help.

[rmottola (Riccardo)]

Joshua, indeed it is python acting up, so it has nothing to do with ICU. Here a transcript:

Artax:~ multix$ python2.7
Python 2.7.17 (default, Oct 23 2019, 23:39:52) 
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import collections
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/collections.py", line 20, in <module>
    from _collections import deque, defaultdict
ImportError: No module named _collections
>>> 

Executing the same command on my other 10.5 Laptop (only difference: x86 vs x86-64) does not have issues.

[rmottola (Riccardo)]

I checked and even if python was build yesterday (check the build date in the previous comment) I have no build log for python27 in /opt/local/var/macports/logs/

Perhaps, since it was considered a successful build, it was removed automatically?

[jmroot (Joshua Root)]

Yes, it would have been. You would have to do another build with sudo port destroot python27.

[rmottola (Riccardo)]

I redid that, build log attached

[jmroot (Joshua Root)]

Hmm, no obvious problems in the log, and _collections.so seems to have been put in the right place in the destroot. Let's check assumptions: does /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_collections.so exist on your system currently?

[rmottola (Riccardo)]

no, it does not exist, instead a file called _collections_failed.so exists, which is not confidence inspiring.

the file is 23rd October 2019, so from one of the latest builds.

Actually, the majority of the files there are renamed "failed":

$ ls /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload
ColorPicker.so                  _Win.so                         _testcapi_failed.so
MacOS.so                        _bisect_failed.so               array_failed.so
Nav.so                          _bsddb_failed.so                audioop_failed.so
OSATerminology.so               _codecs_cn_failed.so            autoGIL_failed.so
Python-2.7.17-py2.7.egg-info    _codecs_hk_failed.so            binascii_failed.so
_AE.so                          _codecs_iso2022_failed.so       bz2_failed.so
_AH.so                          _codecs_jp_failed.so            cPickle_failed.so
_App.so                         _codecs_kr_failed.so            cStringIO_failed.so
_CF_failed.so                   _codecs_tw_failed.so            cmath_failed.so
_CG_failed.so                   _collections_failed.so          crypt_failed.so
_CarbonEvt.so                   _csv_failed.so                  datetime_failed.so
_Cm.so                          _ctypes_failed.so               dbm_failed.so
_Ctl.so                         _ctypes_test_failed.so          fcntl_failed.so
_Dlg.so                         _curses_failed.so               future_builtins_failed.so
_Drag.so                        _curses_panel_failed.so         gestalt.so
_Evt.so                         _elementtree_failed.so          grp_failed.so
_File.so                        _functools_failed.so            icglue.so
_Fm.so                          _hashlib_failed.so              itertools_failed.so
_Folder.so                      _heapq_failed.so                math_failed.so
_Help.so                        _hotshot_failed.so              mmap_failed.so
_IBCarbon.so                    _io_failed.so                   nis_failed.so
_Icn.so                         _json_failed.so                 operator_failed.so
_Launch_failed.so               _locale_failed.so               parser_failed.so
_List.so                        _lsprof_failed.so               pyexpat_failed.so
_Menu.so                        _multibytecodec_failed.so       readline_failed.so
_Mlte.so                        _multiprocessing_failed.so      resource_failed.so
_OSA.so                         _random_failed.so               select_failed.so
_Qd.so                          _scproxy_failed.so              strop_failed.so
_Qdoffs.so                      _sha256_failed.so               syslog_failed.so
_Qt.so                          _sha512_failed.so               termios_failed.so
_Res.so                         _socket_failed.so               time_failed.so
_Scrap.so                       _sqlite3_failed.so              unicodedata_failed.so
_Snd.so                         _ssl_failed.so                  zlib_failed.so
_TE.so                          _struct_failed.so

[jmroot (Joshua Root)]

So it looks like if you just uninstall and then install again it will be fixed. Maybe some temporary problem when you built previously? I'd like to fix it so that won't happen again, but it's hard to do that without the ability to reproduce the problem.

[rmottola (Riccardo)]

the issue... is worse. I uninstalled all python versions, install cmake and first python gets pulled in, built and installed (and has fine shared objects) then it does the same process again, but in +universal (it is a 64bit machine and I enabled that) and now the modules are broken. When I did the test you asked, I did not add +universal...

now, scrolling back in the console I see messages like this:

building '_sha256' extension
/usr/bin/gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -pipe -Os -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Mac/Include -I. -IInclude -I./Include -I/opt/local/include -I/opt/local/include/db48 -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Include -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17 -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Modules/sha256module.c -o build/temp.macosx-10.5-x86_64-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Modules/sha256module.o
/usr/bin/gcc-4.2 -bundle -undefined dynamic_lookup -L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db48 build/temp.macosx-10.5-x86_64-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Modules/sha256module.o -L/opt/local/lib -L/opt/local/lib/db48 -o build/lib.macosx-10.5-x86_64-2.7/_sha256.so
*** WARNING: renaming "_sha256" since importing it failed: dlopen(build/lib.macosx-10.5-x86_64-2.7/_sha256.so, 2): no suitable image found.  Did find:
        build/lib.macosx-10.5-x86_64-2.7/_sha256.so: mach-o, but wrong architecture

and this is repeated the same for every .so

[kencu (Ken)]

Same thing happens for me, trying to build python27 +universal on 10.5 Intel. Log attached.

[jmroot (Joshua Root)]

The -arch flags are clearly being lost somewhere along the way, but I have no immediate idea where or why.

[jmroot (Joshua Root)]

Duplicate #59947 shows this affects 10.6 as well (but apparently not always).

[fhgwright (Fred Wright)]

It may be that this isn't *directly* OS-dependent at all. I encountered this trying to build icu +universal, which thought (incorrectly) that it needed python27 +universal. In the case of icu, the python27 dependency seems to be because it needs Python >=2.7 to build, which isn't met by the Apple Python until 10.7. Of course propagating +universal to a *build* dependency is completely silly, but that's what base currently seems to do.

Since it's probably unusual for users to specifically request +universal for Python, it's likely that it mainly arises from gratuitous universality in build procedures, explaining why the impact isn't wider.

[devernay (Frédéric Devernay)]

it seems like the lib-dynload modules are compiled twice, once with the arch flags and once without. The second time it is built seems to be during the install phase. 10.6 here, building on i386:

building 'itertools' extension
/opt/local/bin/clang-mp-3.4 -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -pipe -Os -arch x86_64 -arch i386 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Mac/Include -I. -IInclude -I./Include -I/opt/local/include -I/opt/local/include/db48 -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Include -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17 -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Modules/itertoolsmodule.c -o build/temp.macosx-10.6-intel-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Modules/itertoolsmodule.o
/opt/local/bin/clang-mp-3.4 -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db48 -arch x86_64 -arch i386 build/temp.macosx-10.6-intel-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Modules/itertoolsmodule.o -L/opt/local/lib -L/opt/local/lib/db48 -o build/lib.macosx-10.6-intel-2.7/itertools.so

...

building 'itertools' extension
/opt/local/bin/clang-mp-3.4 -fno-strict-aliasing -fno-common -dynamic -pipe -Os -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Mac/Include -I. -IInclude -I./Include -I/opt/local/include -I/opt/local/include/db48 -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Include -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17 -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Modules/itertoolsmodule.c -o build/temp.macosx-10.6-i386-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Modules/itertoolsmodule.o
/opt/local/bin/clang-mp-3.4 -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db48 -arch x86_64 -arch i386 build/temp.macosx-10.6-i386-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Modules/itertoolsmodule.o -L/opt/local/lib -L/opt/local/lib/db48 -o build/lib.macosx-10.6-i386-2.7/itertools.so
ld: warning: ignoring file build/temp.macosx-10.6-i386-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.17/Modules/itertoolsmodule.o, file was built for unsupported file format which is not the architecture being linked (i386)
*** WARNING: renaming "itertools" since importing it failed: dynamic module does not define init function (inititertools)

@kencu maybe you could pin python27 to 2.7.16_2 on 10.5 and 10.6 (LeopardPorts and SnowLeopardPorts) until we have a solution? That version doesn't seem to be affected (checked the binary packages from ​http://packages.macports.org/python27/ and the modules dont have the _failed.so suffix). ​http://packages.macports.org/python27/python27-2.7.17_0+universal.darwin_10.i386-x86_64.tbz2 is broken.

[jmroot (Joshua Root)]

As far as I can tell, the python27 binary for 10.6 no longer has this issue:

% lipo -info ./python27-2.7.18_2+universal.darwin_10.i386-x86_64/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_collections.so
Architectures in the fat file: ./python27-2.7.18_2+universal.darwin_10.i386-x86_64/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_collections.so are: x86_64 i386 

Do you still see the issue on 10.5?

[jmroot (Joshua Root)]

No response; assuming fixed.