openssh @9.4p1: build fails with zlib @1.3 installed

[fabianwenk (Fabian Wenk)]

During port upgrade OpenSSH failed during config with this error (it is build from source because I have activated the ldns variant):

checking for deflate in -lz... yes
checking for possibly buggy zlib... yes
configure: error: *** zlib too old - check config.log ***
Your reported zlib version has known security problems.  It's possible your
vendor has fixed these problems without changing the version number.  If you
are sure this is the case, you can disable the check by running
"./configure --without-zlib-version-check".
If you are in doubt, upgrade zlib to version 1.2.3 or greater.
See http://www.gzip.org/zlib/ for details.
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_openssh/openssh/work/openssh-9.4p1" && ./configure --prefix=/opt/local --with-ssl-dir=/opt/local --sysconfdir=/opt/local/etc/ssh --with-privsep-path=/var/empty --with-md5-passwords --with-pid-dir=/opt/local/var/run --with-pam --mandir=/opt/local/share/man --with-zlib=/opt/local --without-kerberos5 --with-libedit --with-pie --with-xauth=/opt/local/bin/xauth --with-ldns --with-audit=bsm --with-keychain=apple 
Exit code: 1
Error: Failed to configure openssh: consult /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_openssh/openssh/work/openssh-9.4p1/config.log
Error: Failed to configure openssh: configure failure: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_openssh/openssh/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there
is a bug.

I did the upgrades with 'port upgrade outdated', zlib 1.2.13_0 -> 1.3_0 and OpenSSH 9.3p2_0 -> 9.4p1_0 and the update of zlib was done before openssh:

 # port installed | grep '^  zlib'
  zlib @1.2.13_0 requested_variants='' platform='darwin 22' archs='x86_64' date='2023-07-21T19:42:11+0200'
  zlib @1.3_0 (active) requested_variants='' platform='darwin 22' archs='x86_64' date='2023-08-18T18:40:38+0200'

relevant parts out of /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_openssh/openssh/work/openssh-9.4p1/config.log:

This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by OpenSSH configure Portable, which was
generated by GNU Autoconf 2.71.  Invocation command line was

  $ ./configure --prefix=/opt/local --with-ssl-dir=/opt/local --sysconfdir=/opt/local/etc/ssh --with-privsep-path=/var/empty --with-md5-passwords --with-pid-dir=/opt/local/var/run --with-pam --mandir=/opt/local/share/man --with-zlib=/opt/local --without-kerberos5 --with-libedit --with-pie --with-xauth=/opt/local/bin/xauth --with-ldns --with-audit=bsm --with-keychain=apple

[...]

configure:10755: checking for zlib
configure:10763: result: yes
configure:10768: checking for zlib.h
configure:10768: /opt/local/bin/clang-mp-15 -c -pipe -Os -isysroot/Applications/
Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.s
dk -arch x86_64 -pipe -Wunknown-warning-option -Qunused-arguments -Wall -Wpointe
r-arith -Wuninitialized -Wsign-compare -Wformat-security -Wsizeof-pointer-memacc
ess -Wno-pointer-sign -Wno-unused-result -Wmisleading-indentation -Wbitwise-inst
ead-of-logical -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -ftrapv -fzero-call-used
-regs=used -fno-builtin-memset -fstack-protector-strong -I/opt/local/include -I/
opt/local/include -DBROKEN_STRNVIS=1 -D__APPLE_SANDBOX_NAMED_EXTERNAL__ -D__APPL
E_API_STRICT_CONFORMANCE -D__APPLE_LAUNCHD__ -isysroot/Applications/Xcode.app/Co
ntents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.sdk conftest.
c >&5
configure:10768: $? = 0
configure:10768: result: yes

[...]

configure:10809: result: yes
configure:10871: checking for possibly buggy zlib
configure:10911: /opt/local/bin/clang-mp-15 -o conftest -pipe -Os -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.sdk -arch x86_64 -pipe -Wunknown-warning-option -Qunused-arguments -Wall -Wpointer-arith -Wuninitialized -Wsign-compare -Wformat-security -Wsizeof-pointer-memaccess -Wno-pointer-sign -Wno-unused-result -Wmisleading-indentation -Wbitwise-instead-of-logical -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -ftrapv -fzero-call-used-regs=used -fno-builtin-memset -fstack-protector-strong -I/opt/local/include -I/opt/local/include -DBROKEN_STRNVIS=1 -D__APPLE_SANDBOX_NAMED_EXTERNAL__ -D__APPLE_API_STRICT_CONFORMANCE -D__APPLE_LAUNCHD__ -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.sdk -L/opt/local/lib -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-search_paths_first -Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.sdk -arch x86_64 -fstack-protector-strong conftest.c -lz  >&5
configure:10911: $? = 0
configure:10911: ./conftest
configure:10911: $? = 1
configure: program exited with status 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "OpenSSH"
| #define PACKAGE_TARNAME "openssh"
| #define PACKAGE_VERSION "Portable"
| #define PACKAGE_STRING "OpenSSH Portable"
| #define PACKAGE_BUGREPORT "openssh-unix-dev@mindrot.org"

[...]

 | #define HAVE_BASENAME 1
| #define WITH_ZLIB 1
| #define HAVE_LIBZ 1
| /* end confdefs.h.  */
| 
| #include <stdio.h>
| #include <stdlib.h>
| #include <zlib.h>
| 
| int
| main (void)
| {
| 
|       int a=0, b=0, c=0, d=0, n, v;
|       n = sscanf(ZLIB_VERSION, "%d.%d.%d.%d", &a, &b, &c, &d);
|       if (n != 3 && n != 4)
|               exit(1);
|       v = a*1000000 + b*10000 + c*100 + d;
|       fprintf(stderr, "found zlib version %s (%d)\n", ZLIB_VERSION, v);
| 
|       /* 1.1.4 is OK */
|       if (a == 1 && b == 1 && c >= 4)
|               exit(0);
| 
|       /* 1.2.3 and up are OK */
|       if (v >= 1020300)
|               exit(0);
| 
|       exit(2);
| 
|   ;
|   return 0;
| }
configure:10916: result: yes
configure:10919: error: *** zlib too old - check config.log ***
Your reported zlib version has known security problems.  It's possible your
vendor has fixed these problems without changing the version number.  If you
are sure this is the case, you can disable the check by running
"./configure --without-zlib-version-check".
If you are in doubt, upgrade zlib to version 1.2.3 or greater.
See http://www.gzip.org/zlib/ for details.

Doing a 'port activate zlib @1.2.13_0' and then doing the 'port update openssh' could configure and build OpenSSH. And even after 'port activate zlib @1.3_0' the OpenSSH cli tools still work.

Bugreport at OpenSSH: ​https://bugzilla.mindrot.org/show_bug.cgi?id=3604

[artkiver (グレェ)]

Thank you for reporting this and going the extra mile to report it upstream as well! I did see zlib 1.3 update the last time I ran port upgrade outdated moments ago, but I am guessing it didn't rebuild OpenSSH for me at least as my install still seems to be working for the time being. I'll start to delve into this more deeply.

[artkiver (グレェ)]

After a preliminary set of experiments, I am certainly able to reproduce an error of the sort below:

checking for library containing basename... none required
checking for zlib... yes
checking for zlib.h... yes
checking for deflate in -lz... yes
checking for possibly buggy zlib... yes
configure: error: *** zlib too old - check config.log ***
Your reported zlib version has known security problems.  It's possible your
vendor has fixed these problems without changing the version number.  If you
are sure this is the case, you can disable the check by running
"./configure --without-zlib-version-check".
If you are in doubt, upgrade zlib to version 1.2.3 or greater.
See http://www.gzip.org/zlib/ for details.

As suggested, make does seem to complete if appending

 --without-zlib-version-check

to the ./configure parameters, but that doesn't really seem as if it is an ideal fix.

I yoinked the following:

#define HAVE_BASENAME 1
#define WITH_ZLIB 1
#define HAVE_LIBZ 1
 /* end confdefs.h.  */
 
 #include <stdio.h>
 #include <stdlib.h>
 #include <zlib.h>
 
 int
 main (void)
 {
 
       int a=0, b=0, c=0, d=0, n, v;
       n = sscanf(ZLIB_VERSION, "%d.%d.%d.%d", &a, &b, &c, &d);
       if (n != 3 && n != 4)
               exit(1);
       v = a*1000000 + b*10000 + c*100 + d;
       fprintf(stderr, "found zlib version %s (%d)\n", ZLIB_VERSION, v);
 
       /* 1.1.4 is OK */
       if (a == 1 && b == 1 && c >= 4)
               exit(0);
 
       /* 1.2.3 and up are OK */
       if (v >= 1020300)
               exit(0);
 
       exit(2);
 
   ;
   return 0;
 }

And compiled and ran it as follows:

% cc zlibcheck.c 
% ./a.out 
found zlib version 1.2.11 (1021100)

Which, isn't really what I was expecting. Perhaps it's checking a different zlib.h than the 1.3 version installed via MacPorts?

For reference, here is some output as related to zlib versions I have installed on one of my systems via MacPorts at present:

port installed |grep zlib
  zlib @1.2.13_0
  zlib @1.3_0 (active)

I'll keep exploring, but before going much further I felt it was worthwhile to share:

1. confirmation of your bug (yay for reproducibility?)
2. preliminary perspective into how it may be sourcing zlib.h for its version check.

[artkiver (グレェ)]

I couldn't help but notice this:

./configure --prefix=/opt/local --with-ssl-dir=/opt/local --sysconfdir=/opt/local/etc/ssh --with-privsep-path=/var/empty --with-md5-passwords --with-pid-dir=/opt/local/var/run --with-pam --mandir=/opt/local/share/man --with-zlib=/opt/local --without-kerberos5 --with-libedit --with-pie --with-xauth=/opt/local/bin/xauth --with-ldns --with-audit=bsm --with-keychain=apple          

Specifically this parameter stood out:

 --with-zlib=/opt/local 

And invocation of configure manually on the OpenSSH 9.4p1 tarball yields the same zlib error.

However, this does not:

./configure --prefix=/opt/local --with-ssl-dir=/opt/local --sysconfdir=/opt/local/etc/ssh --with-privsep-path=/var/empty --with-md5-passwords --with-pid-dir=/opt/local/var/run --with-pam --mandir=/opt/local/share/man --with-zlib=/opt/local/include/zlib.h --without-kerberos5 --with-libedit --with-pie --with-xauth=/opt/local/bin/xauth --with-ldns --with-audit=bsm --with-keychain=apple 

Which, since I don't have the ldns stuff on the system with which I am testing yields the following error instead:

checking for ldns-config... no
checking for ldns support... no
configure: error: ** Incomplete or missing ldns libraries.

Ignoring that for the time being, if I run make with the

--with-zlib=/opt/local/include/zlib.h 

Path declaration as above it does generate the following warning:

ld: warning: -L path '/opt/local/include/zlib.h' is not a directory

But otherwise seems to build without issue?

Meanwhile, from the Portfile the pertinent line is:

                        --with-zlib=${prefix} \

Which, I think typically expands to /opt/local/

I'm not sure it would be wise to change that to something more like --with-zlib=${prefix}/include or --with-zlib=${prefix}/include/zlib.h though I'll do some additional experimenting.

[artkiver (グレェ)]

Deriving some inspiration from the configure.args mentioned here:

​https://trac.macports.org/ticket/26103

Perhaps this would be a better line to have in the Portfile?

                        --with-zlib=${prefix}/lib \

Manually invoking configure as follows:

./configure --prefix=/opt/local --with-ssl-dir=/opt/local --sysconfdir=/opt/local/etc/ssh --with-privsep-path=/var/empty --with-md5-passwords --with-pid-dir=/opt/local/var/run --with-pam --mandir=/opt/local/share/man --with-zlib=/opt/local/lib --without-kerberos5 --with-libedit --with-pie --with-xauth=/opt/local/bin/xauth --with-ldns --with-audit=bsm --with-keychain=apple

The notable change being:

--with-zlib=/opt/local/lib

Also seems to step around the zlib errors otherwise generated. Bonus: also doesn't generate the "ld: warning: -L path '/opt/local/include/zlib.h' is not a directory" warning either!

Let me do some additional experiments locally to a modified Portfile and exploring variants and see if that improves things across the board.

[artkiver (グレェ)]

Drat, that seems as if it was wishful thinking on my part.

I updated the Portfile with

                        --with-zlib=${prefix}/lib \

invocation of port -v install still failed with the following:

configure: error: *** zlib too old - check config.log ***
Your reported zlib version has known security problems.  It's possible your
vendor has fixed these problems without changing the version number.  If you
are sure this is the case, you can disable the check by running
"./configure --without-zlib-version-check".
If you are in doubt, upgrade zlib to version 1.2.3 or greater.
See http://www.gzip.org/zlib/ for details.

[artkiver (グレェ)]

Changing the Portfile with configure.args as follows:

                        --with-zlib=${prefix}/include/ \

                        --with-zlib=${prefix}/include/zlib.h \

Even as explicit as:

                        --with-zlib=/opt/local/include/zlib.h \

Still yielded the "configure: error: * zlib too old - check config.log *" error.

I'm a bit at a loss why manual invocations with a modified --with-zlib= parameter seem to avoid the error, yet as invoked via MacPorts (which appears to be correctly expanding the parameter as I modify it in the Portfile) is yielding different results.

[artkiver (グレェ)]

Oh, groovy. I just checked the upstream BugZilla report and it appears as if they have addressed this!

Pertinent commit here: ​https://github.com/openssh/openssh-portable/commit/cb4ed12ffc332d1f72d054ed92655b5f1c38f621

Salient comment here:

​https://bugzilla.mindrot.org/show_bug.cgi?id=3604#c1

I suppose, temporarily I could create a configure.ac.patch for 9.4p1 and increment the Rev by 1.

Let me do some testing and see if that fixes things locally before submitting a PR.

[artkiver (グレェ)]

diff with zlib 1.3 version configure.ac patch from upstream snapshot

[artkiver (グレェ)]

Preliminary tests with the diff I just attached for configure.ac from the upstream snapshot seem to be going OK.

I'll continue testing variants and submit a PR ASAP!

[artkiver (グレェ)]

Perfunctory testing of variants seems to be going smoothly so far (with the known caveat that Kerberos5 still seems happier with OpenSSL3 for the time being instead of libressl/libressl-devel [I haven't looked at the PRs hoping to change that for a week or two and last time I did they still weren't too happy]).

At any rate, I submitted a PR which rectifies this here:

​https://github.com/macports/macports-ports/pull/19985

That should at least help MacPorts users until the upstream OpenSSH project releases a new version with the included fix.

Thanks!

[artkiver (グレェ)]

The above PR was closed and superseded with the following adding in some comments as suggested by reneeotten:

​https://github.com/macports/macports-ports/pull/19989

Replying to artkiver:

Perfunctory testing of variants seems to be going smoothly so far (with the known caveat that Kerberos5 still seems happier with OpenSSL3 for the time being instead of libressl/libressl-devel [I haven't looked at the PRs hoping to change that for a week or two and last time I did they still weren't too happy]).

At any rate, I submitted a PR which rectifies this here:

​https://github.com/macports/macports-ports/pull/19985

That should at least help MacPorts users until the upstream OpenSSH project releases a new version with the included fix.

Thanks!

[artkiver (グレェ)]

Most recent PR attempt here: ​https://github.com/macports/macports-ports/pull/19991

Build bots are still not happy and my patience for a Friday with hours spent on this is basically burnt to a crisp.

Others are more than welcome to take a crack at this.

I have plans early Saturday morning and will probably look at this some more if I don't feel full of frustration and rage and calm down, but right now I am beyond over it.

[artkiver (グレェ)]

In 764ef7aaa910c84fc70e8215220e7220928ce954/macports-ports (master):

openssh: fix for zlib 1.3 from upstream

Closes: #67986

Co-authored-by: Ryan Schmidt <ryandesign@…>