ncdu @1.12_0: livecheck fails with 403 Forbidden

[breun (Nils Breunese)]

MacPorts says the ncdu livecheck URL returns 403 Forbidden:

$ port -d livecheck ncdu
DEBUG: Changing to port directory: /opt/local/var/macports/sources/rsync.macports.org/release/ports/sysutils/ncdu
DEBUG: OS darwin/16.0.0 (Mac OS X 10.12) arch i386
DEBUG: Going to use alternate build prefix: /Users/breun/.macports
DEBUG: workpath = /Users/breun/.macports/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_sysutils_ncdu/ncdu/work
DEBUG: adding the default universal variant
DEBUG: Reading variant descriptions from /opt/local/var/macports/sources/rsync.macports.org/release/ports/_resources/port1.0/variant_descriptions.conf
DEBUG: Running callback portconfigure::add_automatic_compiler_dependencies
DEBUG: Finished running callback portconfigure::add_automatic_compiler_dependencies
DEBUG: Running callback portbuild::add_automatic_buildsystem_dependencies
DEBUG: Finished running callback portbuild::add_automatic_buildsystem_dependencies
DEBUG: Starting logging for ncdu
DEBUG: Logging disabled, error opening log file: can't create directory "/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_sysutils_ncdu": permission denied
DEBUG: Executing org.macports.main (ncdu)
DEBUG: livecheck phase started at Wed Oct  5 18:23:00 CEST 2016
DEBUG: Executing org.macports.livecheck (ncdu)
DEBUG: Portfile modification date is Wed Oct 05 05:30:05 CEST 2016
DEBUG: Port (livecheck) version is 1.12
DEBUG: Loading the defaults from '/opt/local/var/macports/sources/rsync.macports.org/release/ports/_resources/port1.0/livecheck/fallback.tcl'
DEBUG: Fetching https://dev.yorhel.nl/download/
Error: cannot check if ncdu was updated (The requested URL returned error: 403 Forbidden)

Strangely enough I can request that URL just fine using curl.

[ryandesign (Ryan Carsten Schmidt)]

I'm not able to reproduce this problem.

$ port -d livecheck ncdu
DEBUG: Copying /Users/rschmidt/Library/Preferences/com.apple.dt.Xcode.plist to /opt/local/var/macports/home/Library/Preferences
DEBUG: Changing to port directory: /Users/rschmidt/macports/dports/sysutils/ncdu
DEBUG: OS darwin/15.6.0 (Mac OS X 10.11) arch i386
DEBUG: adding the default universal variant
DEBUG: Reading variant descriptions from /Users/rschmidt/macports/dports/_resources/port1.0/variant_descriptions.conf
DEBUG: Requested variant -atlas is not provided by port ncdu.
DEBUG: Requested variant +accelerate is not provided by port ncdu.
DEBUG: Requested variant +bash_completion is not provided by port ncdu.
DEBUG: Executing variant universal provides universal
DEBUG: Running callback portconfigure::add_automatic_compiler_dependencies
DEBUG: Finished running callback portconfigure::add_automatic_compiler_dependencies
DEBUG: Running callback portbuild::add_automatic_buildsystem_dependencies
DEBUG: Finished running callback portbuild::add_automatic_buildsystem_dependencies
DEBUG: Starting logging for ncdu
DEBUG: Executing org.macports.main (ncdu)
DEBUG: livecheck phase started at Wed Oct  5 16:42:47 CDT 2016
DEBUG: Executing org.macports.livecheck (ncdu)
DEBUG: Port (livecheck) version is 1.9
DEBUG: Loading the defaults from '/Users/rschmidt/macports/dports/_resources/port1.0/livecheck/sourceforge.tcl'
DEBUG: Fetching http://sourceforge.net/projects/ncdu/rss
DEBUG: The regex is "ncdu-(\d\.\d).tar.gz"
DEBUG: The regex matched "ncdu-1.9.tar.gz", extracted "1.9"
DEBUG: The regex matched "ncdu-1.9.tar.gz", extracted "1.9"
DEBUG: The regex matched "ncdu-1.9.tar.gz", extracted "1.9"
DEBUG: The regex matched "ncdu-1.9.tar.gz", extracted "1.9"
DEBUG: The regex matched "ncdu-1.9.tar.gz", extracted "1.9"
DEBUG: The regex matched "ncdu-1.9.tar.gz", extracted "1.9"
DEBUG: The regex matched "ncdu-1.9.tar.gz", extracted "1.9"
DEBUG: The regex matched "ncdu-1.9.tar.gz", extracted "1.9"
DEBUG: The regex matched "ncdu-1.9.tar.gz", extracted "1.9"
DEBUG: The regex matched "ncdu-1.9.tar.gz", extracted "1.9"
DEBUG: The regex matched "ncdu-1.8.tar.gz", extracted "1.8"
DEBUG: The regex matched "ncdu-1.8.tar.gz", extracted "1.8"
DEBUG: The regex matched "ncdu-1.8.tar.gz", extracted "1.8"
DEBUG: The regex matched "ncdu-1.8.tar.gz", extracted "1.8"
DEBUG: The regex matched "ncdu-1.8.tar.gz", extracted "1.8"
DEBUG: The regex matched "ncdu-1.8.tar.gz", extracted "1.8"
DEBUG: The regex matched "ncdu-1.8.tar.gz", extracted "1.8"
DEBUG: The regex matched "ncdu-1.8.tar.gz", extracted "1.8"
DEBUG: The regex matched "ncdu-1.8.tar.gz", extracted "1.8"
DEBUG: The regex matched "ncdu-1.8.tar.gz", extracted "1.8"
DEBUG: The regex matched "ncdu-1.7.tar.gz", extracted "1.7"
DEBUG: The regex matched "ncdu-1.7.tar.gz", extracted "1.7"
DEBUG: The regex matched "ncdu-1.7.tar.gz", extracted "1.7"
DEBUG: The regex matched "ncdu-1.7.tar.gz", extracted "1.7"
DEBUG: The regex matched "ncdu-1.7.tar.gz", extracted "1.7"
DEBUG: The regex matched "ncdu-1.6.tar.gz", extracted "1.6"
DEBUG: The regex matched "ncdu-1.6.tar.gz", extracted "1.6"
DEBUG: The regex matched "ncdu-1.6.tar.gz", extracted "1.6"
DEBUG: The regex matched "ncdu-1.6.tar.gz", extracted "1.6"
DEBUG: The regex matched "ncdu-1.6.tar.gz", extracted "1.6"
DEBUG: The regex matched "ncdu-1.5.tar.gz", extracted "1.5"
DEBUG: The regex matched "ncdu-1.5.tar.gz", extracted "1.5"
DEBUG: The regex matched "ncdu-1.5.tar.gz", extracted "1.5"
DEBUG: The regex matched "ncdu-1.5.tar.gz", extracted "1.5"
DEBUG: The regex matched "ncdu-1.5.tar.gz", extracted "1.5"
DEBUG: The regex matched "ncdu-1.4.tar.gz", extracted "1.4"
DEBUG: The regex matched "ncdu-1.4.tar.gz", extracted "1.4"
DEBUG: The regex matched "ncdu-1.4.tar.gz", extracted "1.4"
DEBUG: The regex matched "ncdu-1.4.tar.gz", extracted "1.4"
DEBUG: The regex matched "ncdu-1.4.tar.gz", extracted "1.4"
DEBUG: The regex matched "ncdu-1.3.tar.gz", extracted "1.3"
DEBUG: The regex matched "ncdu-1.3.tar.gz", extracted "1.3"
DEBUG: The regex matched "ncdu-1.3.tar.gz", extracted "1.3"
DEBUG: The regex matched "ncdu-1.3.tar.gz", extracted "1.3"
DEBUG: The regex matched "ncdu-1.3.tar.gz", extracted "1.3"
DEBUG: The regex matched "ncdu-1.2.tar.gz", extracted "1.2"
DEBUG: The regex matched "ncdu-1.2.tar.gz", extracted "1.2"
DEBUG: The regex matched "ncdu-1.2.tar.gz", extracted "1.2"
DEBUG: The regex matched "ncdu-1.2.tar.gz", extracted "1.2"
DEBUG: The regex matched "ncdu-1.2.tar.gz", extracted "1.2"
ncdu seems to be up to date
DEBUG: Checking time since last reclaim run

Maybe there was a temporary server problem.

[breun (Nils Breunese)]

Looks like you're not using the most recent version of the Portfile. The current version no longer looks at SourceForge, since the developer doesn't use that anymore.

[ryandesign (Ryan Carsten Schmidt)]

Yes, sorry, ignore what I said above.

I don't understand why the problem is happening.

[raimue (Rainer Müller)]

I added a bunch of debug code to our curl wrapper in pextlib to extract the real body from the failed request:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /download/
on this server.<br />
Reason: The client software did not provide a hostname using Server Name Indication (SNI), which is required to access this server.<br />
</p>
</body></html>

This is a problem due to these conditions:

• curl with SecureTransport will not send SNI in TLS client hello, when it was asked not to verify the server certificate (​upstream issue)
• livecheck runs with --ignore-ssl-cert because livecheck.ignore_sslcert yes is the default

I committed a quickfix for this problem in r153620.

However, I would also recommend we make livecheck.ignore_sslcert no the default in base.

[ryandesign (Ryan Carsten Schmidt)]

Replying to raimue@…:

However, I would also recommend we make livecheck.ignore_sslcert no the default in base.

Totally agree. Looking through the history, looks like it's been "no" ever since it was added in r70975 for #17063.

[raimue (Rainer Müller)]

The default has already been changed long time ago in [3de5fafc89ff51f1fb5261ae947aa9fc5abc656b/macports-base].

[raimue (Rainer Müller)]

In 3f3b544904b87fab844168c9505e925d14aa1044/macports-ports:

Revert "ncdu: enforce SSL cert verification in livecheck"

The default for livecheck.ignore_sslcert was changed with 2.4.0,
so this override is no longer necessary.

This reverts commit e928be3d088841120548ee48f3033b4dd3fb0d82.

See: 3de5fafc89ff51f1fb5261ae947aa9fc5abc656b/macports-base
Closes: #52515