Opened 3 years ago

Closed 3 years ago

#63730 closed defect (fixed)

python27: importing ctypes fails on Monterey / Arm64

Reported by: jonwilts Owned by: jmroot (Joshua Root)
Priority: Normal Milestone:
Component: ports Version: 2.7.1
Keywords: monterey Cc:
Port: python27 libffi

Description

Upgraded my M1 Mini to Monterey, and following the Migration instructions for macports. No idea if this is Arm64 only.

During Migration, py27-setuptools fails to build:

:info:build Executing:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-setuptools/py27-setuptools/work/setuptools-44.1.1" && /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 setup.py --no-user-cfg build
:debug:build system:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-setuptools/py27-setuptools/work/setuptools-44.1.1" && /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 setup.py --no-user-cfg build
:info:build Traceback (most recent call last):
:info:build   File "setup.py", line 9, in <module>
:info:build     import setuptools
:info:build   File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-setuptools/py27-setuptools/work/setuptools-44.1.1/setuptools/__init__.py", line 20, in <module>
:info:build     from setuptools.dist import Distribution, Feature
:info:build   File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-setuptools/py27-setuptools/work/setuptools-44.1.1/setuptools/dist.py", line 36, in <module>
:info:build     from setuptools import windows_support
:info:build   File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-setuptools/py27-setuptools/work/setuptools-44.1.1/setuptools/windows_support.py", line 2, in <module>
:info:build     import ctypes
:info:build   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 7, in <module>
:info:build     from _ctypes import Union, Structure, Array
:info:build ImportError: No module named _ctypes
:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-setuptools/py27-setuptools/work/setuptools-44.1.1" && /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 setup.py --no-user-cfg build
:info:build Exit code: 1
:error:build Failed to build py27-setuptools: command execution failed
:debug:build Error code: CHILDSTATUS 33447 1
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$callback {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec -callback portprogress::target_progress_callback build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"
:error:build See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-setuptools/py27-setuptools/main.log for details.

py27-setuptools is a dependency for a number of other ports including py27-libxml2 which seems to be used a lot.

Attachments (3)

p27.log (2.9 MB) - added by jonwilts 3 years ago.
python27 build log
arm64-ffi.h (13.9 KB) - added by jonwilts 3 years ago.
/opt/local/include/arm64-ffi.h
arm64-ffi.2.h (13.9 KB) - added by jonwilts 3 years ago.
/opt/local/include/arm64-ffi.h

Change History (13)

comment:1 Changed 3 years ago by jmroot (Joshua Root)

Milestone: MacPorts Future
Port: python27 added; py27-setuptools removed
Summary: py27-setuptools @ 44.1.1_0.darwin_21.noarch.tbz2 - Fails to install on Monterey / Arm64python27: importing ctypes fails on Monterey / Arm64

Looks like _ctypes may not have been built. The build log for python27 may have information as to why.

comment:2 Changed 3 years ago by jonwilts

Ok - removed and reinstalled python27 with -v. You were right:

building '_ctypes' extension
creating build/temp.macosx-12.0-arm64-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/darwin
creating build/temp.macosx-12.0-arm64-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx
creating build/temp.macosx-12.0-arm64-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/x86
creating build/temp.macosx-12.0-arm64-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/powerpc
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -arch arm64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I_ctypes/darwin -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Mac/Include -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/include -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/powerpc -I. -IInclude -I./Include -I/opt/local/include -I/opt/local/include/db48 -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Include -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18 -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/_ctypes.c -o build/temp.macosx-12.0-arm64-2.7/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/_ctypes.o -DUSING_MALLOC_CLOSURE_DOT_C=1 -DMACOSX
In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/_ctypes.c:113:
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/include/ffi.h:65:3: error: "Unsupported MacOS X CPU type"
#       error "Unsupported MacOS X CPU type"
        ^
In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/_ctypes.c:113:
In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/include/ffi.h:73:
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/include/ffitarget.h:12:2: error: "Unsupported CPU type"
#error "Unsupported CPU type"
 ^
In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/_ctypes.c:113:
In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/include/ffi.h:74:
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/include/fficonfig.h:51:2: error: "Unknown CPU type"
#error "Unknown CPU type"
 ^
In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/_ctypes.c:113:
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/include/ffi.h:171:5: error: unknown type name 'ffi_abi'
                                ffi_abi         abi;
                                ^
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/include/ffi.h:193:2: error: unknown type name 'ffi_sarg'
        ffi_sarg        sint;
        ^
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/include/ffi.h:194:2: error: unknown type name 'ffi_arg'
        ffi_arg         uint;
        ^
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/libffi_osx/include/ffi.h:307:11: error: unknown type name 'ffi_abi'
                                                                                ffi_abi                 abi,
                                                                                ^
In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/_ctypes.c:129:
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18/Modules/_ctypes/ctypes.h:99:5: error: unknown type name 'ffi_closure'
    ffi_closure *pcl_write; /* the C callable, writeable */
    ^
8 errors generated.

Python build finished, but the necessary bits to build these modules were not found:
bsddb185           dl                 imageop         
linuxaudiodev      ossaudiodev        spwd            
sunaudiodev                                           
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


Failed to build these modules:
_ctypes            _Qt                                

comment:3 Changed 3 years ago by jmroot (Joshua Root)

It shouldn't be using the included copy of libffi in the first place. Please just attach the whole log.

comment:4 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)

FYI there was no problem building python27 and all the py27 modules needed to run the buildbot worker on our M1 Mac mini.

Changed 3 years ago by jonwilts

Attachment: p27.log added

python27 build log

Changed 3 years ago by jonwilts

Attachment: arm64-ffi.h added

/opt/local/include/arm64-ffi.h

Changed 3 years ago by jonwilts

Attachment: arm64-ffi.2.h added

/opt/local/include/arm64-ffi.h

comment:5 Changed 3 years ago by jonwilts

I noticed at the top of the log this message

Header file /opt/local/include/ffi.h does not define LIBFFI_H or ffi_wrapper_h 

/opt/local/include/ffi.h contains this:

#ifdef __arm64__
#include "arm64-ffi.h"
#else
#include "ppcintel-ffi.h"
#endif

I've added my /opt/local/include/arm64-ffi.h in case that's useful.

comment:6 Changed 3 years ago by jmroot (Joshua Root)

Looks like some third-party version of libffi is installed in the MacPorts prefix. The libffi port doesn't install a file called arm64-ffi.h. Or if it did, it must be a universal build gone wrong somehow?

Last edited 3 years ago by jmroot (Joshua Root) (previous) (diff)

comment:7 Changed 3 years ago by jmroot (Joshua Root)

Port: libffi added

comment:8 Changed 3 years ago by jmroot (Joshua Root)

In 9719fb3cf851e6c7888a1a855079a76b3f3a0364/macports-ports (master):

python27: libffi detection fixes

Fix getting libffi include dir from pkgconfig. Fix ffi.h validation to
recognise muniversal builds.

See: #63730

comment:9 Changed 3 years ago by jmroot (Joshua Root)

Owner: set to jmroot
Status: newaccepted

It seems that that is just what the headers installed by libffi +universal look like, due to tricks employed by the muniversal portgroup. Please try out the committed fix.

comment:10 Changed 3 years ago by jmroot (Joshua Root)

Resolution: fixed
Status: acceptedclosed

Assuming fixed in the absence of any report to the contrary. Please reopen if not.

Note: See TracTickets for help on using tickets.