py-mitmproxy 10.1 missing dependencies

[breiter (Brian Reiter)]

The py-mitmproxy package has an unsatisfied package dependency on netlib which is a current port. Installing netlib with pip resolves the problem below but then it complains that it also needs flask.

## As installed by MacPorts ##

$ mitmproxy
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/mitmproxy", line 3, in <module>
    from libmproxy import proxy, console, cmdline
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/proxy.py", line 4, in <module>
    from netlib import tcp, http, certutils, http_status, http_auth
ImportError: No module named netlib
$ mitmproxy
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/mitmproxy", line 3, in <module>
    from libmproxy import proxy, console, cmdline
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/proxy.py", line 4, in <module>
    from netlib import tcp, http, certutils, http_status, http_auth
ImportError: No module named netlib
$ mitmproxy
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/mitmproxy", line 3, in <module>
    from libmproxy import proxy, console, cmdline
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/proxy.py", line 4, in <module>
    from netlib import tcp, http, certutils, http_status, http_auth
ImportError: No module named netlib
$ mitmproxy
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/mitmproxy", line 3, in <module>
    from libmproxy import proxy, console, cmdline
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/proxy.py", line 4, in <module>
    from netlib import tcp, http, certutils, http_status, http_auth
ImportError: No module named netlib

## After sudo pip install netlib ##

$ mitmproxy
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/mitmproxy", line 3, in <module>
    from libmproxy import proxy, console, cmdline
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/proxy.py", line 5, in <module>
    import utils, flow, version, platform, controller
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/flow.py", line 11, in <module>
    import app
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/app.py", line 1, in <module>
    import flask
ImportError: No module named flask

## After sudo pip install flask ##

Note that flask installs dependent packages: flask, Werkzeug, Jinja2, itsdangerous, markupsafe.

After all these pip packages are manually installed, mimtproxy runs successfully.

[petrrr]

Cc Me!

[petrrr]

The following dependencies (declared in the setup.py file) are missing in the port: netlib, requests>=1.2.2, flask. In particular, there is no port for netlib yet.

There are more requirements documented in requirements.txt:

Flask>=0.9
Jinja2>=2.7
MarkupSafe>=0.18
Pillow>=2.3.0,<2.4
Werkzeug>=0.8.3
lxml>=3.2.1
netlib>=0.10
pathod>=0.9.2
pyOpenSSL>=0.14
pyasn1>=0.1.7
requests>=1.2.2
urwid>=1.1.1
wsgiref>=0.1.2
jsbeautifier>=1.4.0
cssutils>=1.0,<1.1
pydivert>=0.0.2

Not sure if all this is really accurate. But the Pillow>=2.3.0,<2.4 requirement might be an issue as Macports provides py-Pillow @2.5.1. Also an issue: Macports currently provides py-cssutils @0.9.9.

Brian, can you provide patches and portfiles?

[petrrr]

Port py-netlib added in r123781.

[petrrr]

Note that the port py-flask is available in MacPorts, so if using the Macports' Python versions you should install the port, otherwise you risk to encounter problems, due to conflicts between Macports and pip.

I will add the missing dependencies now, so that they install automatically.

But you should clean up the pip installed packages again, otherwise you risk further problems!

[petrrr]

Dependencies fixed in r123783.

[petrrr]

The requirement specification is fully consistent, so there are minor doubts what is valid. Please test and report if you observe further issues. Thanks!

[breiter (Brian Reiter)]

I ran into a conflict running port upgrade py-mitmproxy between py27-pil and py27-pillow. py27-mitmproxy declares an explicit dependency on py27-pillow but has an implicit one on py27-pil. The former is a fork of the later.

Solution required port uninstall py-mitmproxy py27-mitmproxy py27-pil. And then port install py-mitmproxy.

With that out of the way, it seems to be working.

[petrrr]

Yes, the problem is that earlier versions used py-pil as dependency. This has changed with version 0.0.5 which explicitly now requires Pillow instead of PIL, but this was not considered during update to 0.0.6. I corrected this, but if py-pil is already installed you see the conflict and have to uninstall manually.

Unfortunately, I do not see a way to provide a smoother upgrade path here. This problem should go away at some point.

[petrrr]

So I close this ticket. It looks okay and I have asked for clarifications on the dependencies to the authors. In case there you should still observe some issue, just compile a new ticket (or reopen).