Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#61190 closed defect (fixed)

py38-dns-lexicon @3.4.3_0 certbot execution dns-lexicon missing

Reported by: ArtKoKo Owned by: Schamschula (Marius Schamschula)
Priority: Normal Milestone:
Component: ports Version: 2.6.3
Keywords: Cc:
Port: py-dns-lexicon

Description (last modified by mf2k (Frank Schima))

certbot @1.8.0_0+python38 (active)
certbot-dns-ovh @1.8.0_0+python38 (active)
py38-dns-lexicon @3.4.3_0 (active)

simple execution : sudo certbot

An unexpected error occurred: pkg_resources.DistributionNotFound: The 'dns-lexicon>=2.7.14' distribution was not found and is required by the application

Attachments (1)

log.txt (2.2 KB) - added by ArtKoKo 4 years ago.
log of certbot

Download all attachments as: .zip

Change History (29)

Changed 4 years ago by ArtKoKo

Attachment: log.txt added

log of certbot

comment:1 Changed 4 years ago by mf2k (Frank Schima)

Description: modified (diff)
Keywords: certbot lexicon removed
Owner: set to Schamschula
Status: newassigned

In the future, please add the port maintainer(s) to Cc (port info --maintainers certbot-dns-ovh), if any.

comment:2 Changed 4 years ago by mf2k (Frank Schima)

Description: modified (diff)

comment:3 in reply to:  1 Changed 4 years ago by ArtKoKo

Replying to mf2k:

In the future, please add the port maintainer(s) to Cc (port info --maintainers certbot-dns-ovh), if any.

Thanks. Noted

comment:4 Changed 4 years ago by Schamschula (Marius Schamschula)

Port: py-dns-lexicon added; certbot-dns-ovh removed

Hmm. certbot itself does not require dns-lexicon.

However, as I have certbot-dns-namecheap, and thus py-dns-lexicon, installed I see the same error.

The problem is with py-dns-lexicon, as it currently doesn't install any useful files.

Looking into it.

comment:5 Changed 4 years ago by Schamschula (Marius Schamschula)

As of the 3.4.0 release dns-lexicon uses poetry. I'm not familiar with poetry.

comment:6 in reply to:  4 Changed 4 years ago by ArtKoKo

Replying to Schamschula:

Hmm. certbot itself does not require dns-lexicon.

However, as I have certbot-dns-namecheap, and thus py-dns-lexicon, installed I see the same error.

The problem is with py-dns-lexicon, as it currently doesn't install any useful files.

Looking into it.

Indeed all the ports certbot-dns-* depend on py-dns-lexicon.
a possible improvement ?

comment:7 Changed 4 years ago by ArtKoKo

I'm not familiar myself with poetry but it's yet another package manager to facilitate dependency management : https://python-poetry.org/docs

dgilman can maybe help us? https://ports.macports.org/maintainer/github/dgilman/

comment:8 Changed 4 years ago by Schamschula (Marius Schamschula)

No, certbot does not need certbot-dns-namecheap to be installed. When I deactivated certbot-dns-namecheap and then reinstalled certbot, certbot worked correctly. However, this will loose functionality for some users.

I think the best route is to back py-dns-lexicon up to the last pre-poetry version, i.e. 3.3.28.

comment:9 Changed 4 years ago by ArtKoKo

Yescerbot does not need py-dns-lexicon but cerbot-dns-* does. I agree to come back to the previous version but how ?

comment:10 Changed 4 years ago by Schamschula (Marius Schamschula)

In 981f414eeb62fcef482a322444f942e87dcbdf86/macports-ports (master):

py-dns-lexicon: revert to 3.3.28

we don't want to use poetry to build
remove github PG, let python PG do the work
See: #61190

comment:11 Changed 4 years ago by ArtKoKo

For info, the response of the author of lexicon :

In fact you can continue to use your workflow with pip/setuptools as before, and not care about poetry if you want, thanks to two mechanisms available here:
recent versions of pip (>=10.0), implements PEP 517 and PEP 518, which is a convention to prepare in isolation the build system required in the poetry.toml of a project. Basically you can checkout the project, run pip install lexicon, and pip will fetch poetry on its own to install the project. There is a limitation for editable installations (pip install -e) which fall in the next point for older versions of pip, or for editable installations, an installation through setuptools is required and still possible. Indeed a shim for setup.py has been designed by the packaging community for retro-compatibility purpose, and I added it to Lexicon. In this case setuptools will kick in and you will not benefit from Poetry features, like advanced dependency graph resolution, but for your case I think it will be sufficient.

comment:12 Changed 4 years ago by Schamschula (Marius Schamschula)

We don't use pip in our installation process. Just like poetry, pip manages python projects and their dependencies in parallel to MacPorts. This is a bad idea.

Using setuptools, nothing gets installed, other that a few stub files:

port contents py38-dns-lexicon
Port py38-dns-lexicon contains:
  /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/UNKNOWN-0.0.0-py3.8.egg-info/PKG-INFO
  /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/UNKNOWN-0.0.0-py3.8.egg-info/SOURCES.txt
  /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/UNKNOWN-0.0.0-py3.8.egg-info/dependency_links.txt
  /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/UNKNOWN-0.0.0-py3.8.egg-info/top_level.txt

which, for the most part are empty.

comment:13 Changed 4 years ago by ArtKoKo

Response of AnalogJ/lexicon

No it is me. The shim does not cover the case of old versions of pip, just the case of pip install -e (editable mode). You still need at least pip 10.0+ to make it work through the build isolation. Having a more recent version of pip on FreeBSD is not doable ?

comment:14 Changed 4 years ago by Schamschula (Marius Schamschula)

macOS != FreeBSD. I run FreeBSD on my servers. Fresh Ports != MacPorts. There are major differences how these two port systems work.

Yes, on MacPorts, we have a current version of pip:

port list py38-pip
py38-pip                       @20.2.3         python/py-pip

and poetry:

port list poetry
poetry                         @1.0.3          python/poetry

However, any package installed by pip may conflict with the same package installed by MacPorts. I.e. if I install a package via pip it may install a number dependencies not previously installed by MacPorts. These packages are not recorded in the MacPorts registry. If you then attempt to install the corresponding MacPorts package, the activation phase will fail, as the files will already exist. This is the same issue as with poetry.

For example: before there was a port of tensorflow, I used pip to install tensorflow. Later, when I used MacPorts to install tensorflow, I had to manually uninstall several packages that had been installed by pip.

I go out of my way not to depend on pip to avoid package management issues.

Last edited 4 years ago by Schamschula (Marius Schamschula) (previous) (diff)

comment:15 Changed 4 years ago by ArtKoKo

Response of AnalogJ/lexicon:

Also, depending of the use case:

if it is for development, you should anyway use a Python virtual environment, and in it you will be able to use an up-to-date version of pip
if it is to use it, the wheel is better than installing from sources https://files.pythonhosted.org/packages/4b/3b/dcaa4f6a0930703c73d44e3c6c7520132a353a640180b9da1b5b3436ccb6/dns_lexicon-3.4.3-py3-none-any.whl

comment:16 Changed 4 years ago by ArtKoKo

It's a long shot...

comment:17 Changed 4 years ago by Schamschula (Marius Schamschula)

This port is not for development: py-dns-lexicon is a dependency of certbot-dns-namecheap which is used for production instances of certbot.

MacPorts users want their certbot certificates to be automatically maintained.

comment:18 Changed 4 years ago by ArtKoKo

Hopefully in the future the modules certbot-dns-* will use something other than py-dns-lexicon because the developer is taking a wrong path for us...

comment:19 Changed 4 years ago by ArtKoKo

Question at AnalogJ/lexicon:

I spoke with one of the maintainers of the package manager MacPorts:
The problem is that lexicon is a dependency of the certbot-dns extensions of certbot and that adding another package manager to handle the dependencies of the dependency obviously becomes counterproductive...

Response of AnalogJ/lexicon:

Of course, but MacPorts maintainers need to find a solution for packages moving to PEP 571, because more and more packages will move to pyproject.toml since it is a standard now.
I looked at MacPorts behavior, but failed to understand how a Python package is actually installed. I suppose it is something like python setup.py install, which is exclusive to setuptools. For a maximum compatibility I would recommend to use pip: in fact, pip would not be used as package manager here, but as the build frontend to install the package, with a command like pip install --no-deps. In this case pip will leverage the actual installation to setuptools for legacy, and the defined build system for PEP 517 compliant projects.

comment:20 Changed 4 years ago by ArtKoKo

Going back to the old version seems to bring another missing port:
pkg_resources.DistributionNotFound: The 'dnspython<2requests[security]' distribution was not found and is required by dns-lexicon

murphy's law ?

comment:21 Changed 4 years ago by Schamschula (Marius Schamschula)

Nope.

Just a package that I no longer use myself. I created the certbot port before all the plug-ins were added. In the meantime, I moved both my external servers to FreeBSD.

I keep updating the packages with new releases, but only do very basic functionality tests, as my build machine only runs an intranet web server.

As to the error, I didn't see it myself, when invoking certbot.

A quick search yields https://bugs.archlinux.org/task/67480

so I need to add py-dnspython to the dependencies of py-dns-lexicon.

comment:22 Changed 4 years ago by ArtKoKo

It's fine.
I add py-dnspython and it works very well \\ Thank you Schamschula !

comment:23 Changed 4 years ago by Schamschula (Marius Schamschula)

In 60c1ef5b5a511d8b960e2ad4b576ae33cf77e607/macports-ports (master):

py-dns-lexicon: add py-dnspython dependency

See: #61190

comment:24 Changed 4 years ago by ArtKoKo

Hi Schamschula,

for info, a new dependency appear py-filelock :

pkg_resources.DistributionNotFound: The 'filelock>=3.0.8' distribution was not found and is required by tldextract

Works well after intallation...

comment:25 Changed 4 years ago by Schamschula (Marius Schamschula)

In 0f2c198a531b60807f76932ecad31a4575b7b2b1/macports-ports (master):

py-tldextract: add py-filelock dependancy

See: #61190#comment:24

comment:26 Changed 4 years ago by ryandesign (Ryan Carsten Schmidt)

py-dns-lexicon was updated to 3.5.2 and uses the new python.pep517 yes functionality that was added to the python 1.0 portgroup. Does that resolve this issue?

comment:27 Changed 4 years ago by Schamschula (Marius Schamschula)

Resolution: fixed
Status: assignedclosed

comment:28 Changed 4 years ago by ArtKoKo

Yes I have no trouble with the last lexicon version, well done !

Note: See TracTickets for help on using tickets.