xar @1.8.0.501_0: error: implicit declaration of function 'xar_prop_get_expect_notnull' is invalid in C99 [-Werror,-Wimplicit-function-declaration]

[ryandesign (Ryan Carsten Schmidt)]

On Monterey arm64 and x86_64, xar fails to build:

• ​https://build.macports.org/builders/ports-12_arm64-builder/builds/139710/steps/install-port/logs/stdio
• ​https://build.macports.org/builders/ports-12_x86_64-builder/builds/121033/steps/install-port/logs/stdio

src/xar.c:678:11: error: implicit declaration of function 'xar_prop_get_expect_notnull' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        if( 0 != xar_prop_get_expect_notnull((xar_file_t)x, "checksum/offset" ,&value) ) {
                 ^

xar is a dependency of llvm ports so this will prevent the build of any port needing a newer clang.

[Schamschula (Marius Schamschula)]

I am well aware of this!

However, when I rolled back the failing builds to the previous version, the build bot insisted in attempting to build the incorrect version of xar.

I don't see the logic error in the Portfile.

I attempted to fix the very old macOS versions by removing Big Sur and Monterey from the if statements in [9df0a503e8490c0bb7eda2af37c6cfd9df0c9e89/macports-ports] but that changed nothing.

These two OS versions need their own patch file anyway, as there are different implicit function definitions that are missing. The problem is that I have no way of testing on anything older than macOS Sonoma.

[Schamschula (Marius Schamschula)]

Also see: #71151

[ryandesign (Ryan Carsten Schmidt)]

Replying to Schamschula:

However, when I rolled back the failing builds to the previous version, the build bot insisted in attempting to build the incorrect version of xar.

I don't see the logic error in the Portfile.

I'm not sure what you mean. The port versions seem to be what they should be based on the portfile code:

% for osmajor in $(seq 8 24); do printf %2d: $osmajor; port info --version xar os.major=$osmajor; done 
 8:version: 1.8.0.498
 9:version: 1.8.0.498
10:version: 1.8.0.498
11:version: 1.8.0.498
12:version: 1.8.0.498
13:version: 1.8.0.498
14:version: 1.8.0.501
15:version: 1.8.0.501
16:version: 1.8.0.501
17:version: 1.8.0.501
18:version: 1.8.0.501
19:version: 1.8.0.501
20:version: 1.8.0.501
21:version: 1.8.0.501
22:version: 1.8.0.501
23:version: 1.8.0.501
24:version: 1.8.0.501

Buildbot is building xar 501 on macOS 11 and 12 like you're telling it to. Buildbot is not special; the same thing that happens on Buildbot workers happens on user systems as well.

These two OS versions need their own patch file anyway, as there are different implicit function definitions that are missing.

I haven't yet dug into their code but do you know why this differs by OS version?

The problem is that I have no way of testing on anything older than macOS Sonoma.

It is possible to set up virtual machines running older macOS versions, though that of course takes time and disk space to set up and I wouldn't expect anyone to do that.

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign:

src/xar.c:678:11: error: implicit declaration of function 'xar_prop_get_expect_notnull' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        if( 0 != xar_prop_get_expect_notnull((xar_file_t)x, "checksum/offset" ,&value) ) {
                 ^

xar_prop_get_expect_notnull is declared in xar.h and defined in xar.c. xar.c does #include "xar.h" thanks to patch-stdout.diff. Seems like that should be enough.

However, I see that /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/xar/xar.h also exists and it does not declare xar_prop_get_expect_notnull. It seems to be including this file instead of the one included with the xar source. I'm not sure why. The command looks right to me:

/usr/bin/clang -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -arch x86_64  -Iinclude -Iinclude -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/local/include/libxml2 -c src/xar.c -o src/xar.o

I thought -I paths would be looked in first, before -isysroot and -isystem paths. But the log shows it is accessing the file from the SDK:

src/xar.c:186:6: warning: 'xar_open' is deprecated: first deprecated in macOS 12.0 - xar is a deprecated file format and should not be used. [-Wdeprecated-declarations]
        x = xar_open(filename, WRITE);
            ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/include/xar/xar.h:158:7: note: 'xar_open' has been explicitly marked deprecated here

Different OS versions including different versions of xar.h would certainly account for why this problem occurs on some OS versions and not others, but I don't think we want to use the OS version of xar.h on any OS version.

I see that xar.c does also #include "xar_internal.h", before including xar.h. xar_internal.h does this:

#ifdef XARSIG_BUILDING_WITH_XAR
#include "xar.h"
#else
#include <xar/xar.h>
#endif // XARSIG_BUILDING_WITH_XAR

So if XARSIG_BUILDING_WITH_XAR is defined then we wouldn't need to patch xar.c to include xar.h. And if XARSIG_BUILDING_WITH_XAR is not defined then the include of <xar/xar.h> will certainly find the system version, not the one in the xar source.

xar.xcodeproj/project.pbxproj defines XARSIG_BUILDING_WITH_XAR:

				OTHER_CFLAGS = "-DXARSIG_BUILDING_WITH_XAR";

Maybe all we need to do is define that in the portfile as well, since the portfile doesn't build using the Xcode project. I can try it in a moment.

[ryandesign (Ryan Carsten Schmidt)]

Regardless whether I define XARSIG_BUILDING_WITH_XAR or reinstate the part of the patch of xar_internal.h that you removed in the 501 update, I get this error on macOS 12:

lib/archive.h:45:10: fatal error: 'CommonCrypto/CommonDigestSPI.h' file not found
#include <CommonCrypto/CommonDigestSPI.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

I don't have such a header in /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/CommonCrypto on macOS 12 but I don't see it on a macOS 15 machine either. I see a hint in webkit source code that this is an Apple internal header, so we can't use it.

Wherever this header is included, it is only when __APPLE__ is defined. There is a fallback to openssl for other systems. Maybe we need to take that fallback in MacPorts too.

Attaching my work in progress.

[ryandesign (Ryan Carsten Schmidt)]

work in progress

[Schamschula (Marius Schamschula)]

I think I mistakenly uploaded a newer version of patch-src-xar_internal.h.old.diff. I've uploaded the correct version: [a3d54e33fd4716dcfb060c320750d64b1050f533/macports-ports]

Note: for version 501 xar_internal.h has a line

xar_t xar_fdopen_digest_verify(int fd, int32_t flags, void *expected_toc_digest, size_t expected_toc_digest_len) API_AVAILABLE(macos(15.0));

Does that not mean that this is only available with the macOS 15 API, i.e. Xcode 16.x? However, it compiled fine with Xcode 15.4, after I patched the file.

P.S. Yes, I'm aware that I can run older OS versions in virtual machines. I've done so in the past. It isn't a hardware issue (I have a collection of Macs going back to the original item) as much as a time issue: I'm teaching a double load this semester!

[ryandesign (Ryan Carsten Schmidt)]

Replying to Schamschula:

Note: for version 501 xar_internal.h has a line

xar_t xar_fdopen_digest_verify(int fd, int32_t flags, void *expected_toc_digest, size_t expected_toc_digest_len) API_AVAILABLE(macos(15.0));

Does that not mean that this is only available with the macOS 15 API, i.e. Xcode 16.x?

I'm not really familiar with this. I have read ​Marking API Availability in Objective-C. I think it means that xar_fdopen_digest_verify can only be used when the deployment target is 15.0 or later. xar doesn't ever call xar_fdopen_digest_verify. It appears to be just a library function that will be in libxar for others to use if they want. And if they did want to use it, they would have to do so only when the deployment target is 15.0 or later. I'm not sure if one should be able to build this library when the deployment target is less than 15.0.

But that seems orthogonal to the point that the xar build is using the system xar.h header rather than the one that ships with xar because you removed the fix for #64842.

[Schamschula (Marius Schamschula)]

The portion I removed from the patch file is included in the upstream version of src/xar_internal.h in version 501.

[Marius Schamschula <mschamschula@…>]

In 458829893a027c296ee62b2c054d8a4fa641ba66/macports-ports (master):

xar: add missing funcxtion definition to patch-src-xar_internal.h.diff

See: #71162