Dependency libidn could be replaced by libidn2

[ednl (Ewoud Dronkert)]

Ghostscript is installed as a dependency of ImageMagick, and p5.28-net-libidn as a (sub-sub) dependency of git. Both of those have libidn as a dependency. When doing an update, libidn came with this note: "Please be aware that GNU libidn2 is the successor of GNU libidn. It comes with IDNA 2008 and TR46 implementations and also provides a compatibility layer for GNU libidn." Indeed, libidn has no maintainer but libidn2 does and it is also available in the repo.

I don't know how this works, but my guess is that it would be good to update Ghostscript's and git's dependency to include libidn2 instead of libidn? Thanks!

[ryandesign (Ryan Carsten Schmidt)]

According to the libidn2 readme:

This library is backwards (API) compatible with the ​libidn library. Replacing the idna.h header with idn2.h into a program is sufficient to switch the application from IDNA2003 to IDNA2008 as supported by this library.

Marius, does that seem accurate to you? If it is, then indeed it seems that perhaps all ports that depend on libidn should be switched to use libidn2, and then the libidn port could be deleted (after a year of being marked replaced_by libidn2).

It seems like any configuration code to find libidn would also have to be updated to find libidn2 instead, such as by referencing the libidn2 pc file instead of the libidn pc file.

Ideally, each software package that currently uses libidn should be updated by its developers to support libidn2 without us needing to patch it, if they have not already done that. You could investigate the code of each project and file bug reports with those projects that don't already support this.

The ports that currently depend on libidn are:

• cpuminer
• echoping
• elinks-devel
• FileZilla
• finch
• ghostscript
• gloox
• html-xml-utils
• hydra
• Io
• jabber
• jabberd
• knot
• kopete
• libgsasl
• libpurple
• libVLC2
• loudmouth
• lynx
• maildrop
• monotone
• monotone-devel
• mutt
• p5-net-libidn
• pidgin
• podofo
• prosody
• psi
• skipfish
• tin
• VLC2

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign:

• libgsasl

Upstream says using libidn2 is not possible because libgsasl requires stringprep, a function which no longer exists in libidn2: ​https://lists.gnu.org/archive/html/help-gsasl/2021-05/msg00001.html

From the libidn2 info file:

3.5 Stringprep and libidn2
==========================

The original libidn library includes functionality for the stringprep
processing in ‘stringprep.h’.  That functionality was an integral part
of an IDNA2003 implementation, but it does not apply to IDNA2008.
Furthermore, stringprep processing has been replaced by the PRECIS
framework (RFC8264).

   For the reasons above, libidn2 does not implement stringprep or any
other string processing protocols unrelated to IDNA2008.  Applications
requiring the stringprep processing should continue using the original
libidn, and new applications should consider using the PRECIS framework.

So for libgsasl at least, and possibly for other ports that might require stringprep, we cannot remove libidn or migrate them to libidn2.

• loudmouth

I have filed a bug report: ​https://github.com/mcabber/loudmouth/issues/44

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign:

• p5-net-libidn

p5-net-libidn is intended specifically for using libidn. There is already p5-net-libidn2 for using libidn2.

I have filed a bug report with the developers of p5-io-socket-ssl, a port that currently declares a dependency on p5-net-libidn: ​https://github.com/noxxi/p5-io-socket-ssl/issues/106

[Schamschula (Marius Schamschula)]

In bd6a69e3c922d1daf1b5a2cf074f764aec84c3fa/macports-ports (master):

knot: use libidn2, fix path for knot.conf file, add notes

See: #63309

[Schamschula (Marius Schamschula)]

I haven't dug too deeply into it, but knot did cleanly build using libidn2.

[RJVB (René Bertin)]

Replying to ryandesign:

Ideally, each software package that currently uses libidn should be updated by its developers to support libidn2 without us needing to patch it

Were it not for the below, it would have been perfectly possible to install a shim libidn.h file that links to or includes libidn2.h, idem for a libidn.pc file that contains the libidn2 settings.

Replying to ryandesign:

• libgsasl

Upstream says using libidn2 is not possible because libgsasl requires stringprep, a function which no longer exists in libidn2: ​https://lists.gnu.org/archive/html/help-gsasl/2021-05/msg00001.html

...

So for libgsasl at least, and possibly for other ports that might require stringprep, we cannot remove libidn or migrate them to libidn2.

Is that the only thing missing, and how feasible would it be to "backport" this function to libidn2 (possibly as an additional, preferably static library)?

That said, libidn* aren't the smallest packages around but they aren't exactly huge either, and not mutually exclusive. IOW, no hard reason to retire libidn.

PS: why would a PostScript interpreter need an Internet Domain Name library?

[ryandesign (Ryan Carsten Schmidt)]

Replying to RJVB:

Were it not for the below, it would have been perfectly possible to install a shim libidn.h file that links to or includes libidn2.h, idem for a libidn.pc file that contains the libidn2 settings.

If they were truly drop-in replacements, yeah.

So for libgsasl at least, and possibly for other ports that might require stringprep, we cannot remove libidn or migrate them to libidn2.

Is that the only thing missing,

I have no idea.

and how feasible would it be to "backport" this function to libidn2 (possibly as an additional, preferably static library)?

I would not consider it our job to do such a thing. The developers of libidn2 could do that if they felt it appropriate.

That said, libidn* aren't the smallest packages around but they aren't exactly huge either, and not mutually exclusive. IOW, no hard reason to retire libidn.

Agreed. I was just attempting to follow up on the suggestion of the reporter.

PS: why would a PostScript interpreter need an Internet Domain Name library?

According to Ghostscript's configure.ac file: dnl Check for libidn (needed for Unicode password support)

[RJVB (René Bertin)]

Replying to ryandesign:

and how feasible would it be to "backport" this function to libidn2 (possibly as an additional, preferably static library)?

I would not consider it our job to do such a thing. The developers of libidn2 could do that if they felt it appropriate.

Not intending to start a debate here, so long story short, the same could be said about the backport of OpenSSL 1.1 support to Qt4, or even the whole port:legacy-support. Or indeed to patching the code to use libidn2 instead of libidn ;)

In that light, what do we do with this ticket now? Is there any reason to patch code to use libidn2 instead of libidn if we're going to be keeping both versions anyway?

[ednl (Ewoud Dronkert)]

If there is functionality missing from libidn2 despite the note saying that it has a compatibility layer, then my guess is you should simply keep both. Writing your own extensions to libidn2 seems like a drastic solution, untenable in the long run and not needed for now. But I'm just another user. Thanks for looking into this!

[ryandesign (Ryan Carsten Schmidt)]

Replying to RJVB:

Not intending to start a debate here, so long story short, the same could be said about the backport of OpenSSL 1.1 support to Qt4, or even the whole port:legacy-support. Or indeed to patching the code to use libidn2 instead of libidn ;)

qt4 is presumably dead, its developers having moved on to qt5 and qt6. If the maintainer of the qt4 port chose to backport openssl 1.1 support to keep it working, that's great. An alternative to keeping it working might have been to use the openssl10 port. Keeping the qt4 port working is great for all those other old ports that use qt4 and cannot use qt5 or later.

legacy-support gives us an easy way to enable a whole host of newer software to build on older OS versions that don't have certain features. The alternative is filing individual bug reports with each software developer, asking them to add compatibility functions, which often they are unwilling to do due to the age of the OS version in question.

libidn2 on the other hand is actively-developed software. If they've consciously chosen not to implement a feature that was offered in the original libidn, I don't consider it our job to second-guess them about that. It looks like Homebrew embarked on a similar goal of migrating everything from libidn to libidn2 4 years ago and they asked the developers of libidn2 regarding the stringprep problem ​here. There is some discussion there, including the proposal to split the stringprep functionality out of libidn into a new "libstringprep" library, but it seems to be thought that this is not useful, since the functionality already exists in libidn and can be used from there.

So the verdict is libidn must remain for those old programs that use libidn's coincidental stringprep functions for things not related to IDN processing.

Since libidn's IDN processing is based on the obsolete IDN2003 standard, and libidn2 exists to cover the newer IDN2008 standard, there is still value in us moving software that is capable of doing so to libidn2, and there is value in us reporting bugs to projects that don't yet have libidn2 support asking them to add it.

[RJVB (René Bertin)]

Replying to ryandesign:

Since libidn's IDN processing is based on the obsolete IDN2003 standard, and libidn2 exists to cover the newer IDN2008 standard

Sounds to me like a case of "no user serviceable parts inside" (aka fingers off) if you have no idea what that means and/or if you don't know exactly what your port uses the library for (including possible side-effects the code turns out to rely on)...

(And I'd use a similar consideration before reporting the use of an old version of a standard as a bug.)

[ryandesign (Ryan Carsten Schmidt)]

In bb5ec68646ba79e71d6238cb30aa7c436b544465/macports-ports (master):

cpuminer: Update to 2.5.1

Update to 2.5.1 and remove unused dependencies.

See: #63309

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign:

• cpuminer

Doesn't use libidn. Dependency removed.

• echoping

No evidence of libidn2 support. No longer being maintained.

• elinks-devel

No evidence of libidn2 support. Latest stable release in 2009. Latest development release in 2012. Bug reports are accepted via mailing list or Bugzilla but mailing list subscription and Bugzilla links don't work. Attempted to report this to the given email address but it bounced.

• lynx

Support for libidn2 is in lynx 2.9.0dev.7: ​https://lists.nongnu.org/archive/html/lynx-dev/2021-07/msg00009.html

[ryandesign (Ryan Carsten Schmidt)]

I added libidn as a dependency to proftpd. No evidence of libidn2 support. Requested: ​https://github.com/proftpd/proftpd/issues/1286

[ryandesign (Ryan Carsten Schmidt)]

libidn2 support was added to proftpd yesterday so it should be in the next released version.

[cooljeanius (Eric Gallager)]

Replying to ryandesign:

I added libidn as a dependency to proftpd. No evidence of libidn2 support. Requested: ​https://github.com/proftpd/proftpd/issues/1286

That got closed as fixed on proftpd master.

[nickgaya (Nick Gaya)]

Ghostscript uses libidn's stringprep function to normalize Unicode password strings. This functionality is not supported in libidn2, so the upgrade is essentially blocked for this port. See ​https://bugs.ghostscript.com/show_bug.cgi?id=698774

The libidn2 manual states:

The original libidn library includes functionality for the stringprep processing in stringprep.h. That functionality was an integral part of an IDNA2003 implementation, but it does not apply to IDNA2008. Furthermore, stringprep processing has been replaced by the PRECIS framework (RFC8264).

For the reasons above, libidn2 does not implement stringprep or any other string processing protocols unrelated to IDNA2008. Applications requiring the stringprep processing should continue using the original libidn, and new applications should consider using the PRECIS framework.

​https://www.gnu.org/software/libidn/libidn2/manual/html_node/Converting-from-libidn.html

[RJVB (René Bertin)]

Could Ghostscript be patched (easily) to use that precis framework?