py37-mitmproxy @4.0.4_3: error running mitmproxy and mitmweb

[lhaeger (Lothar Haeger)]

After installing py-mitmproxy and it's dependencies, all I get when trying to run mitmproxy/web is:

# mitmweb-3.7
Traceback (most recent call last):
  File "/opt/local/bin/mitmweb-3.7", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3241, in <module>
    @_call_aside
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside
    f(*args, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'ruamel.yaml.clib' distribution was not found and is required by ruamel.yaml

# mitmproxy-3.7 
Traceback (most recent call last):
  File "/opt/local/bin/mitmproxy-3.7", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3241, in <module>
    @_call_aside
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside
    f(*args, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'ruamel.yaml.clib' distribution was not found and is required by ruamel.yaml

Tried cleaning and reinstall, same result (running the mitmproxy binary distribution from ​https://snapshots.mitmproxy.org/4.0.4/mitmproxy-4.0.4-osx.tar.gz works just fine).

Here's the install output:

sudo port install py-mitmproxy                                                                                                                                                                     1 ↵
--->  Computing dependencies for py-mitmproxy
The following dependencies will be installed:
 py37-asn1
 py37-asn1crypto
 py37-blinker
 py37-brotlipy
 py37-certifi
 py37-cffi
 py37-click
 py37-cryptography
 py37-h11
 py37-h2
 py37-hpack
 py37-hyperframe
 py37-kaitaistruct
 py37-ldap3
 py37-mitmproxy
 py37-openssl
 py37-parsing
 py37-passlib
 py37-pycparser
 py37-pyperclip
 py37-ruamel-yaml
 py37-setuptools
 py37-six
 py37-sortedcontainers
 py37-tornado
 py37-urwid
 py37-wsproto
Continue? [Y/n]: y
--->  Fetching distfiles for py37-setuptools
--->  Verifying checksums for py37-setuptools
--->  Extracting py37-setuptools
--->  Configuring py37-setuptools
--->  Building py37-setuptools
--->  Staging py37-setuptools into destroot
--->  Installing py37-setuptools @41.0.1_0
--->  Activating py37-setuptools @41.0.1_0
--->  Cleaning py37-setuptools
--->  Fetching distfiles for py37-asn1
--->  Attempting to fetch pyasn1-0.4.5.tar.gz from http://nue.de.distfiles.macports.org/py-asn1
--->  Verifying checksums for py37-asn1
--->  Extracting py37-asn1
--->  Configuring py37-asn1
--->  Building py37-asn1
--->  Staging py37-asn1 into destroot
--->  Installing py37-asn1 @0.4.5_0
--->  Activating py37-asn1 @0.4.5_0
--->  Cleaning py37-asn1
--->  Fetching distfiles for py37-blinker
--->  Attempting to fetch blinker-1.4.tar.gz from https://files.pythonhosted.org/packages/source/b/blinker
--->  Verifying checksums for py37-blinker
--->  Extracting py37-blinker
--->  Configuring py37-blinker
--->  Building py37-blinker
--->  Staging py37-blinker into destroot
--->  Installing py37-blinker @1.4_0
--->  Activating py37-blinker @1.4_0
--->  Cleaning py37-blinker
--->  Fetching distfiles for py37-pycparser
--->  Attempting to fetch pycparser-2.19.tar.gz from https://files.pythonhosted.org/packages/source/p/pycparser
--->  Verifying checksums for py37-pycparser
--->  Extracting py37-pycparser
--->  Configuring py37-pycparser
--->  Building py37-pycparser
--->  Staging py37-pycparser into destroot
--->  Installing py37-pycparser @2.19_0
--->  Activating py37-pycparser @2.19_0
--->  Cleaning py37-pycparser
--->  Fetching distfiles for py37-cffi
--->  Attempting to fetch cffi-1.12.3.tar.gz from https://files.pythonhosted.org/packages/source/10/f7/3b302ff34045f25065091d40e074479d6893882faef135c96f181a57ed06
--->  Attempting to fetch cffi-1.12.3.tar.gz from https://files.pythonhosted.org/packages/source/c/cffi
--->  Verifying checksums for py37-cffi
--->  Extracting py37-cffi
--->  Applying patches to py37-cffi
--->  Configuring py37-cffi
--->  Building py37-cffi
--->  Staging py37-cffi into destroot
--->  Installing py37-cffi @1.12.3_0
--->  Activating py37-cffi @1.12.3_0
--->  Cleaning py37-cffi
--->  Fetching distfiles for py37-brotlipy
--->  Verifying checksums for py37-brotlipy
--->  Extracting py37-brotlipy
--->  Applying patches to py37-brotlipy
--->  Configuring py37-brotlipy
--->  Building py37-brotlipy
--->  Staging py37-brotlipy into destroot
--->  Installing py37-brotlipy @0.7.0_1
--->  Activating py37-brotlipy @0.7.0_1
--->  Cleaning py37-brotlipy
--->  Fetching distfiles for py37-certifi
--->  Attempting to fetch certifi-2019.6.16.tar.gz from https://files.pythonhosted.org/packages/source/c/certifi
--->  Verifying checksums for py37-certifi
--->  Extracting py37-certifi
--->  Configuring py37-certifi
--->  Building py37-certifi
--->  Staging py37-certifi into destroot
--->  Installing py37-certifi @2019.06.16_0
--->  Activating py37-certifi @2019.06.16_0
--->  Cleaning py37-certifi
--->  Fetching distfiles for py37-click
--->  Attempting to fetch Click-7.0.tar.gz from https://files.pythonhosted.org/packages/source/c/click
--->  Verifying checksums for py37-click
--->  Extracting py37-click
--->  Configuring py37-click
--->  Building py37-click
--->  Staging py37-click into destroot
--->  Installing py37-click @7.0_0
--->  Activating py37-click @7.0_0
--->  Cleaning py37-click
--->  Fetching distfiles for py37-asn1crypto
--->  Attempting to fetch asn1crypto-0.24.0.tar.gz from https://files.pythonhosted.org/packages/source/fc/f1/8db7daa71f414ddabfa056c4ef792e1461ff655c2ae2928a2b675bfed6b4
--->  Attempting to fetch asn1crypto-0.24.0.tar.gz from https://files.pythonhosted.org/packages/source/a/asn1crypto
--->  Verifying checksums for py37-asn1crypto
--->  Extracting py37-asn1crypto
--->  Configuring py37-asn1crypto
--->  Building py37-asn1crypto
--->  Staging py37-asn1crypto into destroot
--->  Installing py37-asn1crypto @0.24.0_0
--->  Activating py37-asn1crypto @0.24.0_0
--->  Cleaning py37-asn1crypto
--->  Fetching distfiles for py37-six
--->  Attempting to fetch six-1.12.0.tar.gz from https://files.pythonhosted.org/packages/source/s/six/
--->  Verifying checksums for py37-six
--->  Extracting py37-six
--->  Configuring py37-six
--->  Building py37-six
--->  Staging py37-six into destroot
--->  Installing py37-six @1.12.0_0
--->  Activating py37-six @1.12.0_0
--->  Cleaning py37-six
--->  Fetching distfiles for py37-cryptography
--->  Attempting to fetch cryptography-2.7.tar.gz from https://files.pythonhosted.org/packages/source/c/cryptography
--->  Verifying checksums for py37-cryptography
--->  Extracting py37-cryptography
--->  Configuring py37-cryptography
--->  Building py37-cryptography
--->  Staging py37-cryptography into destroot
--->  Installing py37-cryptography @2.7_0
--->  Activating py37-cryptography @2.7_0
--->  Cleaning py37-cryptography
--->  Fetching distfiles for py37-hpack
--->  Attempting to fetch hpack-3.0.0.tar.gz from http://nue.de.distfiles.macports.org/py-hpack/3.0.0_1
--->  Verifying checksums for py37-hpack
--->  Extracting py37-hpack
--->  Configuring py37-hpack
--->  Building py37-hpack
--->  Staging py37-hpack into destroot
--->  Installing py37-hpack @3.0.0_0
--->  Activating py37-hpack @3.0.0_0
--->  Cleaning py37-hpack
--->  Fetching distfiles for py37-hyperframe
--->  Attempting to fetch hyperframe-5.1.0.tar.gz from https://files.pythonhosted.org/packages/source/h/hyperframe/
--->  Verifying checksums for py37-hyperframe
--->  Extracting py37-hyperframe
--->  Configuring py37-hyperframe
--->  Building py37-hyperframe
--->  Staging py37-hyperframe into destroot
--->  Installing py37-hyperframe @5.1.0_0
--->  Activating py37-hyperframe @5.1.0_0
--->  Cleaning py37-hyperframe
--->  Fetching distfiles for py37-h2
--->  Attempting to fetch h2-3.0.1.tar.gz from https://files.pythonhosted.org/packages/source/h/h2/
--->  Verifying checksums for py37-h2
--->  Extracting py37-h2
--->  Configuring py37-h2
--->  Building py37-h2
--->  Staging py37-h2 into destroot
--->  Installing py37-h2 @3.0.1_0
--->  Activating py37-h2 @3.0.1_0
--->  Cleaning py37-h2
--->  Fetching distfiles for py37-kaitaistruct
--->  Attempting to fetch kaitaistruct-0.8.tar.gz from https://files.pythonhosted.org/packages/source/k/kaitaistruct
--->  Verifying checksums for py37-kaitaistruct
--->  Extracting py37-kaitaistruct
--->  Configuring py37-kaitaistruct
--->  Building py37-kaitaistruct
--->  Staging py37-kaitaistruct into destroot
--->  Installing py37-kaitaistruct @0.8_0
--->  Activating py37-kaitaistruct @0.8_0
--->  Cleaning py37-kaitaistruct
--->  Fetching distfiles for py37-ldap3
--->  Attempting to fetch ldap3-2.6.tar.gz from http://nue.de.distfiles.macports.org/py-ldap3
--->  Verifying checksums for py37-ldap3
--->  Extracting py37-ldap3
--->  Configuring py37-ldap3
--->  Building py37-ldap3
--->  Staging py37-ldap3 into destroot
--->  Installing py37-ldap3 @2.6_0
--->  Activating py37-ldap3 @2.6_0
--->  Cleaning py37-ldap3
--->  Fetching distfiles for py37-openssl
--->  Attempting to fetch pyopenssl-19.0.0.tar.gz from http://nue.de.distfiles.macports.org/py-openssl
--->  Verifying checksums for py37-openssl
--->  Extracting py37-openssl
--->  Configuring py37-openssl
--->  Building py37-openssl
--->  Staging py37-openssl into destroot
--->  Installing py37-openssl @19.0.0_0
--->  Activating py37-openssl @19.0.0_0
--->  Cleaning py37-openssl
--->  Fetching distfiles for py37-parsing
--->  Attempting to fetch pyparsing-2.2.0.tar.gz from http://nue.de.distfiles.macports.org/py-parsing
--->  Verifying checksums for py37-parsing
--->  Extracting py37-parsing
--->  Configuring py37-parsing
--->  Building py37-parsing
--->  Staging py37-parsing into destroot
--->  Installing py37-parsing @2.2.0_0
--->  Activating py37-parsing @2.2.0_0
--->  Cleaning py37-parsing
--->  Fetching distfiles for py37-passlib
--->  Attempting to fetch passlib-1.7.1.tar.gz from https://files.pythonhosted.org/packages/source/p/passlib/
--->  Verifying checksums for py37-passlib
--->  Extracting py37-passlib
--->  Configuring py37-passlib
--->  Building py37-passlib
--->  Staging py37-passlib into destroot
--->  Installing py37-passlib @1.7.1_0
--->  Activating py37-passlib @1.7.1_0
--->  Cleaning py37-passlib
--->  Fetching distfiles for py37-pyperclip
--->  Attempting to fetch pyperclip-1.6.0.tar.gz from https://files.pythonhosted.org/packages/source/p/pyperclip/
--->  Verifying checksums for py37-pyperclip
--->  Extracting py37-pyperclip
--->  Configuring py37-pyperclip
--->  Building py37-pyperclip
--->  Staging py37-pyperclip into destroot
--->  Installing py37-pyperclip @1.6.0_0
--->  Activating py37-pyperclip @1.6.0_0
--->  Cleaning py37-pyperclip
--->  Fetching distfiles for py37-ruamel-yaml
--->  Attempting to fetch 0.16.0.tar.gz from http://nue.de.distfiles.macports.org/py-ruamel-yaml
--->  Verifying checksums for py37-ruamel-yaml
--->  Extracting py37-ruamel-yaml
--->  Configuring py37-ruamel-yaml
--->  Building py37-ruamel-yaml
--->  Staging py37-ruamel-yaml into destroot
--->  Installing py37-ruamel-yaml @0.16.0_0
--->  Activating py37-ruamel-yaml @0.16.0_0
--->  Cleaning py37-ruamel-yaml
--->  Fetching distfiles for py37-sortedcontainers
--->  Attempting to fetch sortedcontainers-2.1.0.tar.gz from https://files.pythonhosted.org/packages/source/s/sortedcontainers
--->  Verifying checksums for py37-sortedcontainers
--->  Extracting py37-sortedcontainers
--->  Configuring py37-sortedcontainers
--->  Building py37-sortedcontainers
--->  Staging py37-sortedcontainers into destroot
--->  Installing py37-sortedcontainers @2.1.0_0
--->  Activating py37-sortedcontainers @2.1.0_0
--->  Cleaning py37-sortedcontainers
--->  Fetching distfiles for py37-tornado
--->  Attempting to fetch tornado-6.0.3.tar.gz from https://files.pythonhosted.org/packages/source/t/tornado
--->  Verifying checksums for py37-tornado
--->  Extracting py37-tornado
--->  Configuring py37-tornado
--->  Building py37-tornado
--->  Staging py37-tornado into destroot
--->  Installing py37-tornado @6.0.3_0
--->  Activating py37-tornado @6.0.3_0
--->  Cleaning py37-tornado
--->  Fetching distfiles for py37-urwid
--->  Attempting to fetch urwid-2.0.1.tar.gz from http://nue.de.distfiles.macports.org/py-urwid
--->  Verifying checksums for py37-urwid
--->  Extracting py37-urwid
--->  Configuring py37-urwid
--->  Building py37-urwid
--->  Staging py37-urwid into destroot
--->  Installing py37-urwid @2.0.1_0
--->  Activating py37-urwid @2.0.1_0
--->  Cleaning py37-urwid
--->  Fetching distfiles for py37-h11
--->  Attempting to fetch h11-0.8.1.tar.gz from https://files.pythonhosted.org/packages/source/h/h11/
--->  Verifying checksums for py37-h11
--->  Extracting py37-h11
--->  Configuring py37-h11
--->  Building py37-h11
--->  Staging py37-h11 into destroot
--->  Installing py37-h11 @0.8.1_0
--->  Activating py37-h11 @0.8.1_0
--->  Cleaning py37-h11
--->  Fetching distfiles for py37-wsproto
--->  Attempting to fetch wsproto-0.14.0.tar.gz from https://files.pythonhosted.org/packages/source/w/wsproto/
--->  Verifying checksums for py37-wsproto
--->  Extracting py37-wsproto
--->  Configuring py37-wsproto
--->  Building py37-wsproto
--->  Staging py37-wsproto into destroot
--->  Installing py37-wsproto @0.14.0_0
--->  Activating py37-wsproto @0.14.0_0
--->  Cleaning py37-wsproto
--->  Fetching distfiles for py37-mitmproxy
--->  Attempting to fetch mitmproxy-4.0.4.tar.gz from http://nue.de.distfiles.macports.org/py-mitmproxy
--->  Verifying checksums for py37-mitmproxy
--->  Extracting py37-mitmproxy
--->  Applying patches to py37-mitmproxy
--->  Configuring py37-mitmproxy
--->  Building py37-mitmproxy
--->  Staging py37-mitmproxy into destroot
--->  Installing py37-mitmproxy @4.0.4_3
--->  Activating py37-mitmproxy @4.0.4_3
--->  Cleaning py37-mitmproxy
--->  Cleaning py-mitmproxy
--->  Updating database of binaries
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.
--->  Some of the ports you installed have notes:
  py37-mitmproxy has the following notes:
    The onboarding app (for intercepting http://mitm.it and serving MITM
    CA certificates) is disabled by default due to incompatibility with
    Tornado 6.x. Please use the +onboardingapp variant to install older
    Tornado and enable them.

[mf2k (Frank Schima)]

In the future, please add the port maintainer(s) to Cc (port info --maintainers py37-mitmproxy), if any.

[lhaeger (Lothar Haeger)]

Replying to mf2k:

In the future, please add the port maintainer(s) to Cc (port info --maintainers py37-mitmproxy), if any.

Sure, no problem (I assumed naming the affected port(s) would do that automatically).

[petrrr]

I won't have the occasion to look into this soon. If anyone wants to look at this feel free to commit directly.

BTW: If anyone would like to take over maintaining this port it would be fine as well.

[Crosmatron]

ruamel.yaml.clib is a separate package now, so it needs to be installed too. I guess it needs its own port.

As a temporary workaround you can run:

sudo pip-3.7 install ruamel.yaml.clib

[lhaeger (Lothar Haeger)]

I've created ​https://trac.macports.org/ticket/58941 to handle the missing py-ruamel-yaml-clib port for a proper solution.

Until that's available, adding py37-pip as a dependency and running the above command in a pre-build action could be a workaround.

[michaelld (Michael Dickens)]

In 6affcffd8a12b3256fe6c151ed5d8ad5ea89dc07/macports-ports (master):

py-ruamel-yaml-clib: new port

Closes: #58792

[michaelld (Michael Dickens)]

In 2be7f0e8efb15a3ff4784655f83fc7f2b06c6131/macports-ports (master):

py-mitmproxy: add dependency on ruamel-yaml-clib

Really Closes: #58792