py37-matplotlib @3.1.3_0+cairo+webagg: importing fails

[ladavid]

Hi all,

I recently updated py37-matplotlib. I now cannot import and receive the following error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-11-66d69862b80a> in <module>
      4 import numpy as np
      5 import pandas as pa
----> 6 import pylab as pl

/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pylab.py in <module>
----> 1 from matplotlib.pylab import *
      2 import matplotlib.pylab
      3 __doc__ = matplotlib.pylab.__doc__

/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/pylab.py in <module>
    216 import matplotlib as mpl
    217 
--> 218 from matplotlib.dates import (
    219     date2num, num2date, datestr2num, strpdate2num, drange, epoch2num,
    220     num2epoch, mx2num, DateFormatter, IndexDateFormatter, DateLocator,

/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/dates.py in <module>
    154 import matplotlib.units as units
    155 import matplotlib.cbook as cbook
--> 156 import matplotlib.ticker as ticker
    157 
    158 __all__ = ('datestr2num', 'date2num', 'num2date', 'num2timedelta', 'drange',

/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/ticker.py in <module>
    172 from matplotlib import rcParams
    173 from matplotlib import cbook
--> 174 from matplotlib import transforms as mtransforms
    175 
    176 _log = logging.getLogger(__name__)

/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/transforms.py in <module>
     41 
     42 from matplotlib import cbook
---> 43 from matplotlib._path import (
     44     affine_transform, count_bboxes_overlapping_bbox, update_path_extents)
     45 from .path import Path

ImportError: dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/_path.cpython-37m-darwin.so, 2): Symbol not found: _MPL_matplotlib__contour_ARRAY_API
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/_path.cpython-37m-darwin.so
  Expected in: flat namespace
 in /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/_path.cpython-37m-darwin.so

If it helps, I've tried uninstalling and reinstalling the port. Also, I'll confess to having pip installed too on my machine. I've tried to keep package installs separate between pip and mac ports, but feel free to let me know if it looks like matplotlib might be unhappy with a dependency installed by pip. Thank you!

sudo port install py37-matplotlib
--->  Computing dependencies for py37-matplotlib
--->  Fetching archive for py37-matplotlib
Warning: Your DNS servers incorrectly claim to know the address of nonexistent hosts. This may cause checksum mismatches for some ports. See this page for more information: <https://trac.macports.org/wiki/MisbehavingServers>
--->  Attempting to fetch py37-matplotlib-3.1.3_0+cairo+webagg.darwin_19.x86_64.tbz2 from https://packages.macports.org/py37-matplotlib
--->  Attempting to fetch py37-matplotlib-3.1.3_0+cairo+webagg.darwin_19.x86_64.tbz2 from http://aus.us.packages.macports.org/macports/packages/py37-matplotlib
--->  Attempting to fetch py37-matplotlib-3.1.3_0+cairo+webagg.darwin_19.x86_64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/py37-matplotlib
--->  Fetching distfiles for py37-matplotlib
--->  Verifying checksums for py37-matplotlib
--->  Extracting py37-matplotlib
--->  Applying patches to py37-matplotlib
Warning: reinplace s|^cairo=False|cairo=True| didn't change anything in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_python_py-matplotlib/py37-matplotlib/work/matplotlib-3.1.3/setup.cfg
--->  Configuring py37-matplotlib
--->  Building py37-matplotlib
--->  Staging py37-matplotlib into destroot
--->  Installing py37-matplotlib @3.1.3_0+cairo+webagg
--->  Activating py37-matplotlib @3.1.3_0+cairo+webagg
--->  Cleaning py37-matplotlib
--->  Scanning binaries for linking errors
--->  No broken files found.                             
--->  No broken ports found.
--->  Some of the ports you installed have notes:
  py37-matplotlib has the following notes:
    The default backend is the interactive Mac OS X backend. Different backends can be specified using the ~/.matplotlib/matplotlibrc file. More
    details regarding backends can be found in the matplotlib FAQ:
    
    https://matplotlib.org/tutorials/introductory/usage.html#backends}}}

[reneeotten (Renee Otten)]

At least for me on macOS Mojave, doing the import pylab as pl and failing one (from matplotlib._path import (affine_transform, count_bboxes_overlapping_bbox, update_path_extents)) works without problems. I haven't upgrade to Catalina, which you seem to use, so can't comment on whether that's a factor. As a side note, I would certainly not recommend installing packages with both pip and through MacPorts, but let's see if we can figure this out.

Can you please provide the output of port -v installed pkgconfig py37-setuptools python37 freetype libpng qhull py37-cycler py37-dateutil py37-kiwisolver py37-numpy py37-parsing py37-tornado py37-cairo? These are matplotlib's dependencies and it would be good to see what you've installed through MacPorts. Also please check whether you have any of those installed through pip and, if so, which ones are loaded if you do import <package> in Python.

Again, I am not sure this is really a problem with the MacPorts port, at least not on the macOS versions I've tried. It would be good to see if someone with a "clean" MacPorts-only Python install can try and reproduce this - @m2fk can you perhaps try the import statement above and see if that works for you?

[mf2k (Frank Schima)]

I can confirm the issue with the variants listed on Catalina.

[mf2k (Frank Schima)]

FYI it works fine with the +qt5 variant instead of +cairo.

[reneeotten (Renee Otten)]

thank you Frank for checking that! So it seems the issue is with the cairo backend and, again, I don't see this on Mojave which makes troubleshooting difficult for me because I have no way of reproducing it. Having said that, can you please provide the output of port -v installed py37-cairo cairo, jus to make sure you have the default variants as well? In my case this gives:

> port -v installed py37-cairo cairo
The following ports are currently installed:
  cairo @1.16.0_0+quartz+x11 (active) platform='darwin 18' archs='x86_64' date='2019-10-01T20:17:19-0400'
  py37-cairo @1.19.0_0 (active) platform='darwin 18' archs='x86_64' date='2020-01-28T09:56:43-0500'

Secondly, please do sudo port -dv test py37-cairo; that test-suite runs without trouble on my machine let's see if that's the case for you as well.

[ladavid]

Thanks Frank and Renee!

Here is what I get when I run: port -v installed py37-cairo cairo

{{{ The following ports are currently installed:

cairo @1.14.12_0+quartz+x11 platform='darwin 17' archs='x86_64' date='2018-06-21T22:41:12-0400' cairo @1.16.0_0+quartz+x11 (active) platform='darwin 19' archs='x86_64' date='2019-10-17T23:03:08-0400' py37-cairo @1.17.1_0 platform='darwin 17' archs='x86_64' date='2018-10-05T21:11:23-0400' py37-cairo @1.19.0_0 (active) platform='darwin 19' archs='x86_64' date='2020-02-10T21:14:35-0500' }}}

Also, when I run: sudo port -dv test py37-cairo I see:

{{{The following dependencies will be installed:

py37-atomicwrites py37-hypothesis py37-packaging py37-pluggy py37-py py37-pytest py37-sortedcontainers pytest_select

Continue? [Y/n]: }}}

[reneeotten (Renee Otten)]

okay, so you have the latest versions installed; the ones for "darwint 19" are active. Seeing that you still have ports installed (but not active) for "darwin 17" suggests to me that you perhaps recently updated to Catalina, correct? If that's the case did you follow the ​migration steps?

Regarding the tests: I am more interested what happens after you tell MacPorts to continue and actual run the tests ;)

[ladavid]

Of course :) I did continue and the dependencies were installed. Unfortunately, that did not seem to resolve the problem.

Also, I do believe I followed the migration steps.

I'm now trying to build the +qt5 version as Frank suggested. I'll let you know how it works out!

[someuser12]

After upgrading py36-matplotlib under Mac OS 10.11.6 and Mac OS 10.13.6, and after a fresh install of py37-matplotlib and of py37-matplotlib +tkinter (thus without Cairo) under a fresh install of Macports under Mac OS 10.15.3, I see a similar problem: "import matplotlib" fails under all of these installations of Matplotlib, with the same error message Symbol not found: _MPL_matplotlib__image_ARRAY_API.

In the case of py37-matplotlib +tkinter in the fresh install under Mac OS 10.15.3, the complete message is

>>> import matplotlib.pyplot
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/__init__.py", line 205, in <module>
    _check_versions()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/__init__.py", line 190, in _check_versions
    from . import ft2font
ImportError: dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-darwin.so, 2): Symbol not found: _MPL_matplotlib__contour_ARRAY_API
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-darwin.so
  Expected in: flat namespace
 in /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-darwin.so
>>>

It is not clear to me if this linkage issue is related to this ticket. Should I open a new ticket?

So far, I found no workaround (reinstalling py36-matplotlib @3.1.2 instead of the newest 3.1.3 does not solve the problem, which is presumably due to the upgrade of some other package). Numpy and Scipy, in contrast, still work.

[ladavid]

Installing the +qt5 version didn't work for me. Also, building from source (port -v -s) did not work either. I too have not yet found a workaround.

[mf2k (Frank Schima)]

FYI I tried installing cairo-devel to see if that helped, but it did not. Using +qt5 works great for me however.

[someuser12]

No success here under Mac OS 10.15.3 with +qt5:

% port location py37-matplotlib 
Port py37-matplotlib 3.1.3_0+qt5+webagg is installed as an image in:
/opt/local/var/macports/software/py37-matplotlib/py37-matplotlib-3.1.3_0+qt5+webagg.darwin_19.x86_64.tbz2
% python3.7                     
Python 3.7.6 (default, Dec 22 2019, 01:09:06) 
[Clang 11.0.0 (clang-1100.0.33.12)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib.pyplot
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/pyplot.py", line 32, in <module>
    import matplotlib.colorbar
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/colorbar.py", line 27, in <module>
    import matplotlib.artist as martist
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/artist.py", line 13, in <module>
    from .path import Path
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/path.py", line 17, in <module>
    from . import _path, cbook, rcParams
ImportError: dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/_path.cpython-37m-darwin.so, 2): Symbol not found: _MPL_matplotlib__image_ARRAY_API
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/_path.cpython-37m-darwin.so
  Expected in: flat namespace
 in /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/matplotlib/_path.cpython-37m-darwin.so
>>> ^D
% sw_vers 
ProductName:    Mac OS X
ProductVersion: 10.15.3
BuildVersion:   19D76
%

[physicsbeany]

I also just hit this problem after an "upgrade outdated" (several weeks overdue). My OS version is 10.14.6 (Mojave), and the newly installed broken port was py37-matplotlib @3.1.3_0+cairo+webagg

I uninstalled, and activated the previously installed version -- py37-matplotlib @3.1.2_0+cairo+webagg -- which appears to be working.

@reneeotten , would you like me to try some installation/variant tests on my Mojave machine?

[reneeotten (Renee Otten)]

After uninstalling and re-installing (either from source or the binary file) I do now see the same issue on macOS Mojave as well. I am convinced it worked before for me (and py38-matplotlib is still fine), but at least I can reproduce it now - admittedly I still don't know how to fix it though.

[reneeotten (Renee Otten)]

okay, I *think* I have found the issue: it seems related to doing parallel builds. Likely related to the fact that this was ​recently enabled by default in the python PG.

For me adding use_parallel_build no to the Portfile and installing from source solved the problem. Can someone please check if that solves the issue in your case as well? If someone can confirm this I will commit a revised Portfile. Thanks!

[someuser12]

Congratulations, your solution (disabling parallel builds) solves the issue for me too! (Mac OS 10.11.6, Mac OS 10.15.3)

[reneeotten (Renee Otten)]

In e424f96eda64a9ca921b868048a99343b7b731bb/macports-ports (master):

py-matplotlib: disable parallel build

Fixes: #60074