Python loading system numpy in preference to MacPorts version (Lion)

[jamie@…]

I'm having an issue where python scripts are loading the system numpy version instead of the MacPorts compiled version, this leads to the following error message:

RuntimeError: module compiled against API version 6 but this version of numpy is 4

and in some cases (e.g. scripts that use matplotlib) causes crashes.

This has only started occurring on my system since I upgraded to OS X Lion.

The fix so far seems to be to add the following code to the top of each script before loading numpy:

sys.path.reverse()

but this is extremely kludgy fix, and I never had to previously do this. My PYTHONPATH variable points to my local python repository and to the MacPorts site-packages directory only. Is there somewhere else I can tell python to stop loading stuff from /System/Library/Frameworks/Python.framework?

[guygurari@…]

Cc Me!

[guygurari@…]

Same problem here, and the workaround works for me. Thanks!

[jamie@…]

I should add that another way to fix this issue, which I'm now using (modifying all my python scripts which are used across multiple platforms in a mission critical system is not an option for me), is to simply make the Apple installed python-2.7 the default using:

port select python python27-apple

[jmroot (Joshua Root)]

This is my sys.path in macports python2.7:

>>> import sys
>>> print sys.path
['', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PIL', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info']

What is in yours? There must be some system paths if reversing it helps? Which python port are you using anyway?

[ptmkenny@…]

I have the same issue. This is what I get:

>>> import sys
>>> print sys.path
['', '/Library/Python/2.7/site-packages/virtualenv-1.6.4-py2.7.egg', '/Library/Python/2.7/site-packages/CherryPy-3.2.2-py2.7.egg', '/Library/Python/2.7/site-packages/Mnemosyne-2.0_RC4-py2.7.egg', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC', '/Library/Python/2.7/site-packages']

[ptmkenny@…]

Cc Me!

[bardeau]

I am experiencing the same problem for the software I am in charge (​http://www.iram.fr/IRAMFR/GILDAS/).

From what I can understand, it seems that the numpy/core/multiarray.so dynamic library provided by the MacPorts-numpy has been built with the numpy API version 4, while the include files it offers provide the API version 6. API (and ABI) versions can be found in numpy/core/include/numpy/_numpyconfig.h (NPY_API_VERSION and NPY_ABI_VERSION resp).

When you import numpy.core.multiarray (explicitely, or implicitely when importing numpy) from the python interpreter, there is no problem since numpy/core/multiarray.so is self-consistent.

But custom C-based Python-Numpy extensions are built with those lines in particular: #include "numpy/arrayobject.h" import_array();

At load time the function "import_array" checks if the 'NPY_FEATURE_VERSION' (from NPY_API_VERSION in _numpyconfig.h) is consistent with the one numpy/core/multiarray.so has been built with (see PyArray_GetNDArrayCFeatureVersion in numpy/core/code_generators/generate_numpy_api.py in numpy sources). The error message comes from here.

I can also quote those few lines from doc/source/reference/c-api.array.rst: "ABI incompatibility is automatically detected in every numpy's version. API incompatibility detection was added in numpy 1.4.0. If you want to supported many different numpy versions with one extension binary, you have to build your extension with the lowest NPY_FEATURE_VERSION as possible."

So at this stage I do not why the MacPorts-numpy is inconsistent. Is this intended?

[bardeau]

Cc Me!

[Russell-Jones-OxPhys (Russell Jones)]

Cc Me!

[ttimo@…]

Cc Me!

[ttimo@…]

Would appreciate a fix for this. The workaround sys.path.reverse still works though.

[tsutterley (Tyler C Sutterley)]

Cc Me!

[dershow]

Cc Me!

[petrrr]

Cc Me!

[petrrr]

There is nothing py-numpy-specific with this problem, right? It is just about which site-packages have precedence, or that MP's version does not have it for Python 2.7.

[petrrr]

If I understand things correctly, the problem observed here is basically a symptom, due to bug/feature as in ticket #34763.

[larryv (Lawrence Velázquez)]

The likely cause has been fixed in r126840. Could someone verify that it solves this issue?

[jmroot (Joshua Root)]

No response; assuming this was the same issue.