Opened 13 years ago

Closed 10 years ago

#31891 closed defect (duplicate)

Python loading system numpy in preference to MacPorts version (Lion)

Reported by: jamie@… Owned by: macports-tickets@…
Priority: Normal Milestone:
Component: ports Version: 2.0.3
Keywords: Cc: guygurari@…, ptmkenny@…, bardeau, Russell-Jones-OxPhys (Russell Jones), ttimo@…, jyrkiwahlstedt, dh@…, michaelld (Michael Dickens), tsutterley (Tyler C Sutterley), dershow, petrrr, larryv (Lawrence Velázquez)
Port: python27 py-numpy

Description (last modified by ryandesign (Ryan Carsten Schmidt))

I'm having an issue where python scripts are loading the system numpy version instead of the MacPorts compiled version, this leads to the following error message:

RuntimeError: module compiled against API version 6 but this version of numpy is 4

and in some cases (e.g. scripts that use matplotlib) causes crashes.

This has only started occurring on my system since I upgraded to OS X Lion.

The fix so far seems to be to add the following code to the top of each script before loading numpy:

sys.path.reverse()

but this is extremely kludgy fix, and I never had to previously do this. My PYTHONPATH variable points to my local python repository and to the MacPorts site-packages directory only. Is there somewhere else I can tell python to stop loading stuff from /System/Library/Frameworks/Python.framework?

Change History (21)

comment:1 Changed 13 years ago by ryandesign (Ryan Carsten Schmidt)

Description: modified (diff)

comment:2 Changed 13 years ago by guygurari@…

Cc: guygurari@… added

Cc Me!

comment:3 Changed 13 years ago by guygurari@…

Same problem here, and the workaround works for me. Thanks!

comment:4 Changed 13 years ago by jamie@…

I should add that another way to fix this issue, which I'm now using (modifying all my python scripts which are used across multiple platforms in a mission critical system is not an option for me), is to simply make the Apple installed python-2.7 the default using:

port select python python27-apple

comment:5 Changed 12 years ago by jmroot (Joshua Root)

This is my sys.path in macports python2.7:

>>> import sys
>>> print sys.path
['', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PIL', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info']

What is in yours? There must be some system paths if reversing it helps? Which python port are you using anyway?

comment:6 Changed 12 years ago by ptmkenny@…

I have the same issue. This is what I get:

>>> import sys
>>> print sys.path
['', '/Library/Python/2.7/site-packages/virtualenv-1.6.4-py2.7.egg', '/Library/Python/2.7/site-packages/CherryPy-3.2.2-py2.7.egg', '/Library/Python/2.7/site-packages/Mnemosyne-2.0_RC4-py2.7.egg', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC', '/Library/Python/2.7/site-packages']

comment:7 Changed 12 years ago by ptmkenny@…

Cc: ptmkenny@… added

Cc Me!

comment:8 in reply to:  description Changed 12 years ago by bardeau

I am experiencing the same problem for the software I am in charge (http://www.iram.fr/IRAMFR/GILDAS/).

From what I can understand, it seems that the numpy/core/multiarray.so dynamic library provided by the MacPorts-numpy has been built with the numpy API version 4, while the include files it offers provide the API version 6. API (and ABI) versions can be found in numpy/core/include/numpy/_numpyconfig.h (NPY_API_VERSION and NPY_ABI_VERSION resp).

When you import numpy.core.multiarray (explicitely, or implicitely when importing numpy) from the python interpreter, there is no problem since numpy/core/multiarray.so is self-consistent.

But custom C-based Python-Numpy extensions are built with those lines in particular: #include "numpy/arrayobject.h" import_array();

At load time the function "import_array" checks if the 'NPY_FEATURE_VERSION' (from NPY_API_VERSION in _numpyconfig.h) is consistent with the one numpy/core/multiarray.so has been built with (see PyArray_GetNDArrayCFeatureVersion in numpy/core/code_generators/generate_numpy_api.py in numpy sources). The error message comes from here.

I can also quote those few lines from doc/source/reference/c-api.array.rst: "ABI incompatibility is automatically detected in every numpy's version. API incompatibility detection was added in numpy 1.4.0. If you want to supported many different numpy versions with one extension binary, you have to build your extension with the lowest NPY_FEATURE_VERSION as possible."

So at this stage I do not why the MacPorts-numpy is inconsistent. Is this intended?

comment:9 Changed 12 years ago by bardeau

Cc: bardeau@… added

Cc Me!

comment:10 Changed 12 years ago by Russell-Jones-OxPhys (Russell Jones)

Cc: russell.jones@… added

Cc Me!

comment:11 Changed 11 years ago by ttimo@…

Cc: ttimo@… added

Cc Me!

comment:12 Changed 11 years ago by ttimo@…

Would appreciate a fix for this. The workaround sys.path.reverse still works though.

comment:13 Changed 11 years ago by larryv (Lawrence Velázquez)

Cc: jwa@… dh@… michaelld@… added

comment:14 Changed 11 years ago by mf2k (Frank Schima)

Port: python27 py-numpy added

comment:15 Changed 11 years ago by tsutterley (Tyler C Sutterley)

Cc: tsutterley@… added

Cc Me!

comment:16 Changed 11 years ago by dershow

Cc: dersh@… added

Cc Me!

comment:17 Changed 10 years ago by petrrr

Cc: petr@… added

Cc Me!

comment:18 Changed 10 years ago by petrrr

There is nothing py-numpy-specific with this problem, right? It is just about which site-packages have precedence, or that MP's version does not have it for Python 2.7.

comment:19 Changed 10 years ago by petrrr

If I understand things correctly, the problem observed here is basically a symptom, due to bug/feature as in ticket #34763.

comment:20 Changed 10 years ago by larryv (Lawrence Velázquez)

Cc: larryv@… added

The likely cause has been fixed in r126840. Could someone verify that it solves this issue?

comment:21 Changed 10 years ago by jmroot (Joshua Root)

Resolution: duplicate
Status: newclosed

No response; assuming this was the same issue.

Note: See TracTickets for help on using tickets.