matplotlib: fail to import due to a circular import

[mmaclow (Mordecai-Mark Mac Low)]

py310-matplotlib @3.5.1_0+cairo+webagg generates circular import error on import

I have a fresh installation of MacPorts (built yesterday) on a newly upgraded version of MacOS Monterey on an x86 MacBook Pro, with the current version of Xcode. I have installed python310, py310-numpy, and so forth, including py310-matplotlib. Everything works, with no errors generated during install. However when I try to import matplotlib into a jupyter notebook, I get the error message:

ImportError: cannot import name '_c_internal_utils' from partially initialized module 'matplotlib' (most likely due to a circular import) (/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/matplotlib/__init__.py)

I have tried uninstalling and cleaning the port py310-matplotlib, and then reinstalling it, without improvement.

I don't know why it is showing up in the python3.9/site-packages rather than python3.10/site-packages, but I'm not sure whether that is the problem here.

[mmaclow (Mordecai-Mark Mac Low)]

Full call tree

[reneeotten (Renee Otten)]

sorry for the trouble, but it seems to work fine for me... Are you sure that you're using the Python 3.10 version of jupyter-notebook, because that doesn't seem to be the case; you should be able to run it with jupyter-notebook-3.10.

If that still doesn't work, can you please provide the output of port installed | grep -E "matplotlib|jupyter"?

[mmaclow (Mordecai-Mark Mac Low)]

Thanks for checking!

I don't think it's a jupyter issue, as I have just tried using python 3.10 and get the same error:

dust-1793 25 % python
Python 3.10.2 (main, Jan 15 2022, 11:31:35) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/matplotlib/__init__.py", line 109, in <module>
    from . import _api, _version, cbook, docstring, rcsetup
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/matplotlib/cbook/__init__.py", line 31, in <module>
    from matplotlib import _api, _c_internal_utils
ImportError: cannot import name '_c_internal_utils' from partially initialized module 'matplotlib' (most likely due to a circular import) (/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/matplotlib/__init__.py)
>>> 

Here's the list of installed packages requested anyway:

dust-1793 26 % port installed | grep -E "matplotlib|jupyter"
  py-matplotlib @3.5.1_0 (active)
  py310-jupyter @1.0.0_1 (active)
  py310-jupyter_client @7.1.2_0 (active)
  py310-jupyter_console @6.4.0_0 (active)
  py310-jupyter_core @4.7.1_0 (active)
  py310-jupyterlab_pygments @0.1.2_0 (active)
  py310-jupyterlab_widgets @1.0.2_0 (active)
  py310-matplotlib @3.5.1_0+cairo+webagg (active)
  py310-matplotlib-inline @0.1.3_0 (active)

[reneeotten (Renee Otten)]

You should not have the stub port py-matplotlib installed, always specify the pyXY version you want to use. Can you please try and uninstall that one and see if the problem disappears? If you start python 3.10 it should never import something from the 3.9 site-packages…

[mmaclow (Mordecai-Mark Mac Low)]

listings of site-packages/matplotlib for 3.9 and 3.10

[mmaclow (Mordecai-Mark Mac Low)]

Yes, I am afraid that I tried installing the stub port after I got the initial failure looking for a way around it. I have uninstalled it, with no result.

I then went on and uninstalled py310-matplotlib. Very curiously, I get exactly the same error message anyway! I tried uninstalling py39-matplotlib, but nothing was there.

After those uninstalls, I examined the contents of the 3.9 and 3.10 site-packages/matplotlib* directories, and found lots in each one, and very different items. I've attached a listing (again, after an uninstall).

I wonder if some other package still has py39-matplotlib as a dependency and installed it, possibly incorrectly? py310-jupyter is my suspicion, as it seemed to load the 3.9 version of jupyter as well.

(Just to confirm, installing py310-matplotlib makes no difference to the error, although it does restore the many python modules that are still in py39-matplotlib.)

[reneeotten (Renee Otten)]

I suspect you've used python_select to have it start python310 when typing python - correct? Also did you make sure to only install Python packages trough MacPorts, in other words no mixing of stuff with pip and such?

I don't see how any of this can happen. I am pretty sure your problem is related to the fact that Python 3.10 imports packages that were installed for Python 3.9. Nothing in MacPorts will do that for you and it works fine for me, so I bet it is something specific to your system.... Inspect your environment variables for example and make sure you haven't set PYTHONPATH and such.

[mmaclow (Mordecai-Mark Mac Low)]

Yes, I have used python_select as suggested by the python installation.

I did the full reinstall of MacPorts because of previous problems with pip conflicting with it, but I thought I had wiped out my pip installation.

My PYTHONPATH is indeed set. Is that now deprecated? It is the problem, in fact. That is, when I took the 3.9 site packages directory out of it, the problem went away, and matplotlib now works normally.

Thank you for your help with this! I'm teaching a class at Columbia U in numerical astrophysics that relied on making this work again. I apologize for putting this up as a ticket, but I could not figure out how MacPorts was generating this result in a new installation.

[jmroot (Joshua Root)]

Replying to mmaclow:

My PYTHONPATH is indeed set. Is that now deprecated? It is the problem, in fact. That is, when I took the 3.9 site packages directory out of it, the problem went away, and matplotlib now works normally.

It's not deprecated, but as you have discovered, it can easily cause undesired behaviour if you set it incorrectly (much like PATH).