Build error on Ubuntu when libmd-dev is installed: unknown type name ‘SHA2_CTX’ / conflicting types for ‘SHA256_CTX’

[mojca (Mojca Miklavec)]

I tried to build base on Ubuntu. I'll try to provide more information, but here's when it gets stuck for me when building macports base:

cc -c -DUSE_TCL_STUBS -DTCL_NO_DEPRECATED -g -O2 -std=c99 -Wextra -Wall -pedantic     -DHAVE_CONFIG_H -I/path-to/macports-base/src -I/path-to/macports-base/src -I. -I/path-to/macports-base/vendor/vendor-destroot/opt/local/libexec/macports/include -I./../compat -fPIC sha256cmd.c -o sha256cmd.o
In file included from sha256cmd.c:66:0:
/usr/include/sha256.h:34:9: error: unknown type name ‘SHA2_CTX’
 typedef SHA2_CTX SHA256_CTX;
         ^~~~~~~~
/usr/include/sha256.h:34:18: error: conflicting types for ‘SHA256_CTX’
 typedef SHA2_CTX SHA256_CTX;
                  ^~~~~~~~~~
In file included from /usr/include/sha256.h:32:0,
                 from sha256cmd.c:66:
./sha2.h:52:3: note: previous declaration of ‘SHA256_CTX’ was here
 } SHA256_CTX;
   ^~~~~~~~~~
../../Mk/macports.tea.mk:2: recipe for target 'sha256cmd.o' failed
make[2]: *** [sha256cmd.o] Error 1
make[2]: Leaving directory '/home/mo

I suspect that one of the dependencies might be missing. I'll attach more logs later.

[jmroot (Joshua Root)]

Might just be a different version of libmd than was tested with.

[mojca (Mojca Miklavec)]

If I look at (first search hit) ​https://github.com/aosm/libmd/blob/master/sha256.h, I see

#ifdef __APPLE__
#define COMMON_DIGEST_FOR_OPENSSL
#include <CommonCrypto/CommonDigest.h>
#else /* !__APPLE__ */
typedef struct SHA256Context {
        uint32_t state[8];
        uint32_t count[2];
        unsigned char buf[64];
} SHA256_CTX;
#endif /* __APPLE__ */

The file sha256cmd.c contains neither sha256.h nor the else part.

Including that header results in

In file included from /usr/include/sha256.h:32:0,
                 from sha256cmd.c:44:
./sha2.h:49:2: error: unknown type name ‘u_int32_t’
  u_int32_t state[8];
  ^~~~~~~~~

[harens (Haren S)]

I know this ticket was opened two years ago, but on raspbian I found that using libssl-dev rather than libmd fixes it for me.

[Ionic (Mihai Moldovan)]

If you don't have libmd installed, libcrypto (i.e., OpenSSL and derivatives) will be used.

Now, if you do have libmd installed, this library ships a sha256.h header file which includes sha2.h.

The issue is that base also ships an (incompatible) sha2.h header file (which should only be getting used if neither libmd nor libcrypto are available) in src/pextlib1.0, which wrongly gets included by libmd's sha256.h header file, making the build fail.

I'd propose to move pextlib's stand-alone SHA2 implementation (sha2.{c,h}) to src/compat/sha2.

That should fix this. Up until it breaks because a library changes API, but that'll always be a risk.

Speaking of which... there is more breakage.

aclocal.m4 checks for SHA1_File via AC_CHECK_FUNCS, which explicitly requires a symbol the linker can find. Neither FreeBSD's libmd nor the standalone libmd that is getting used on Linux systems define such a symbol - instead, most often it's a macro. So we'll have to switch to AC_CHECK_DECLS instead.

With that, using libmd should finally work again.

Everyone on board with that?

[kencu (Ken)]

I had no problem installing MacPorts on Ubuntu linux 20.04 1 year ago following this recipe I documented at the time, with no changes whatsoever to any MacPorts build code.

InstallingMacPortsOnUbuntuLinux

I just had to force a few things in macports.conf that weren't being automatically detected after installation, as per that recipe.

[kencu (Ken)]

I played around with it for a while, posted a few tickets about fixes in base that could be done to make it work without the macports.conf forces. But to be honest there was just no apparent interest in making this work, so I dropped it. I can't recall if I closed the tickets as "wontfix" or just left them open.

[kencu (Ken)]

Looks like I left them open

60414

60431

60432

[kencu (Ken)]

Just for fun, I tried just now installing the tip of the current github base repo on Ubuntu 20.04, and again it installed right through, without touching any code. I'm just updating the ports tree now, but I presume it will still work the same as it did a year ago.

[Ionic (Mihai Moldovan)]

Ken, the established problem is that it fails to build if the libmd-dev package is installed.

Without that package it will pick libssl-dev, i.e., OpenSSL, and build fine.

Uninstalling libmd-dev temporarily while building base is possible, but a nuisance.

[kencu (Ken)]

Oh, I thought Mojca and harens were just trying to get MacPorts built and running on Ubuntu.

Having it not conflict with other non-default Ubuntu packages that might be installed is indeed a whole new level of complexity that I didn't realize you were interested in working on.

[mojca (Mojca Miklavec)]

Please note that when I experienced the error, I had no clue that I was supposed to uninstall something in order to be able to build macports base. It would be helpful if the configure script would at least tell you that "Hey, I see that you have this library installed. That's not compatible, please use X instead." For me it was just a cryptic error.

[Ionic (Mihai Moldovan)]

You shouldn't have to uninstall anything. It's just a bug in base. I'll fix that. :)

libmd is pretty new in the Linux world, which likely explains why the issue wasn't caught earlier (or why nobody ran into it earlier).

[kencu (Ken)]

and there are probably about 4 people on Earth who have tried to install MP on Ubuntu... there could be many such conflicts :)

[kencu (Ken)]

I updated InstallingMacPortsOnUbuntuLinux to warn people about this issue.

[Ionic (Mihai Moldovan)]

PR up for review at ​https://github.com/macports/macports-base/pull/237.

[Mihai Moldovan <ionic@…>]

In 1e2ff3991b8f50fab0eabacd3b72cd42099f6e83/macports-base (master):

Fix libmd linux (​https://github.com/macports/macports-base/pull/237)

The standalone libmd implementation as used on Linux includes a header
called "sha2.h" which is supposed to be relative to its own include
directory.

Unfortunately, MacPorts ships its own SHA2 implementation in pextlib (in
case neither CommonCrypto, nor libmd nor OpenSSL-like implementations
are available), which also uses the "sha2.h" header file name. This
seems to take precedence and both header files are not compatible, so
the build fails.

Move the standalone SHA2 implementation to src/compat/sha2 and update
the references.

Additionally, SHA1_File is almost never a proper function, but most
often defined as a macro set to the real function. In any case, checking
for it via AC_CHECK_FUNCS() is wrong, so switch to AC_CHECK_DECLS() and
use the updated checking macros.

Fixes: #58060