Opened 12 months ago
Last modified 9 months ago
#68178 assigned defect
rav1e: build fails due to git_allocator missing members — at Version 17
Reported by: | lukaso (Lukas Oberhuber) | Owned by: | mascguy (Christopher Nielsen) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | 2.8.1 |
Keywords: | Cc: | MarcusCalhoun-Lopez (Marcus Calhoun-Lopez), Dave-Allured (Dave Allured) | |
Port: | rav1e |
Description (last modified by ryandesign (Ryan Carsten Schmidt))
First reported on #68026, via @macsguy:
This may require updating the version of the git2 crate [used by rav1e], now that libgit2 has been updated to 1.7.1:
libgit2/src/util/array.h:59:19: error: no member named 'greallocarray' in 'git_allocator' if ((new_array = git__reallocarray(a->ptr, new_size, item_size)) == NULL) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libgit2/src/libgit2/annotated_commit.c:34:21: error: no member named 'gcalloc' in 'git_allocator' annotated_commit = git__calloc(1, sizeof(git_annotated_commit)); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libgit2/src/libgit2/annotated_commit.c:48:34: error: no member named 'gstrdup' in 'git_allocator' annotated_commit->description = git__strdup(description); ^~~~~~~~~~~~~~~~~~~~~~~~
I gave a shot at trying to update the crates, but got errors after updating the crate list:
:info:extract error: failed to select a version for the requirement `git2 = "^0.15"` :info:extract candidate versions found which didn't match: 0.18.0 :info:extract location searched: directory source `/Users/lukasoberhuber/macports-gimp3-arm64/var/macports/build/_Users_lukasoberhuber_project_ports_multimedia_rav1e/rav1e/work/.home/.cargo/macports` (which is replacing registry `crates-io`) :info:extract required by package `built v0.5.2` :info:extract ... which satisfies dependency `built = "^0.5.2"` of package `rav1e v0.6.6 (/Users/lukasoberhuber/macports-gimp3-arm64/var/macports/build/_Users_lukasoberhuber_project_ports_multimedia_rav1e/rav1e/work/rav1e-0.6.6)` :info:extract perhaps a crate was updated and forgotten to be re-vendored? :info:extract As a reminder, you're using offline mode (--offline) which can sometimes cause surprising resolution failures, if this error is too confusing you may wish to retry without the offline flag. :info:extract Command failed: /Users/lukasoberhuber/macports-gimp3-arm64/bin/cargo --offline update :info:extract Exit code: 101
Change History (20)
Changed 12 months ago by lukaso (Lukas Oberhuber)
Attachment: | rav1e_main.log added |
---|
comment:1 Changed 12 months ago by mascguy (Christopher Nielsen)
Cc: | MarcusCalhoun-Lopez added |
---|---|
Description: | modified (diff) |
Owner: | set to mascguy |
Status: | new → assigned |
Summary: | rav1e fails to build on arm64 → rav1e: build fails on arm, due to git_allocator missing members; git2 crate [used by rav1e] may need to be updated |
Version: | → 2.8.1 |
comment:2 Changed 12 months ago by mascguy (Christopher Nielsen)
Keywords: | arm64 added |
---|
comment:3 Changed 12 months ago by rottegift (Sean Doran)
FWIW, I just built successfully with
port -v install rav1e configure.ldflags="-ld_classic -force_load /usr/lib/libiconv.dylib" configure.cflags="-Wl,-ld_classic"
which is ugly, but works up to not-yet-found problems with differing header files
otool -L /opt/local/bin/rav1e /opt/local/bin/rav1e: /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /opt/local/lib/libiconv.2.dylib (compatibility version 9.0.0, current version 9.1.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.0.0)
the -force_load
probably is portable to older macOS/xcode.
I don't think this is the right solution, but if one was, say, in the middle of building something with rav1e as a dependency, this might get the job done.
comment:4 Changed 12 months ago by mascguy (Christopher Nielsen)
Lukas, my thoughts regarding updating the version of the git2 crate may be off in left field, given the following:
The source for libgit2 is included in the libgit2-sys crate so there's no need to pre-install the libgit2 library, the libgit2-sys crate will figure that and/or build that for you.
Given that - combined with the fact that we haven't changed the rav1e
port - we should be fine as-is.
Can you try a clean build of rav1e
, without any prior history/log, to ensure the same results occur?
comment:5 Changed 12 months ago by Dave-Allured (Dave Allured)
Cc: | Dave-Allured added |
---|
comment:6 Changed 12 months ago by lukaso (Lukas Oberhuber)
I did a forced uninstall and a port clean
. If I should do any more clean than that, let me know how.
Attaching the new log.
Changed 12 months ago by lukaso (Lukas Oberhuber)
Attachment: | rav1e_main.2.log added |
---|
Post force uninstall and clean of rav1e
comment:7 Changed 12 months ago by lukaso (Lukas Oberhuber)
OK, this actually happens on x86_64 as well. Attaching the log. Same problem.
Changed 12 months ago by lukaso (Lukas Oberhuber)
Attachment: | rav1e_x86_64_main.log added |
---|
comment:8 Changed 12 months ago by mascguy (Christopher Nielsen)
Summary: | rav1e: build fails on arm, due to git_allocator missing members; git2 crate [used by rav1e] may need to be updated → rav1e: build fails due to git_allocator missing members |
---|
comment:9 Changed 12 months ago by mascguy (Christopher Nielsen)
Keywords: | arm64 removed |
---|
comment:10 follow-up: 11 Changed 12 months ago by lukaso (Lukas Oberhuber)
So it looks like these missing functions were removed in v1.7.1. So that's why we are stuffed. Also meaning they are not using semantic versioning (boo on them). https://github.com/libgit2/libgit2/commit/9dd1bfe81c3aba6ce35ed85bf62fa75d9c61e1a6.
The solution is probably to revert libgit2 in macports. Clearly the build is not pulling the embedded version when it is linking (or so it seems).
comment:11 follow-up: 12 Changed 12 months ago by mascguy (Christopher Nielsen)
Replying to lukaso:
The solution is probably to revert libgit2 in macports. Clearly the build is not pulling the embedded version when it is linking (or so it seems).
No, that's probably not going to happen, unless this is more wide-spread. And given that other Rust-based ports build fine with the latest release of libgit2
- including cargo
and rust
itself - that strongly suggests this is a rav1e
-specific issue.
comment:12 follow-up: 13 Changed 12 months ago by lukaso (Lukas Oberhuber)
Replying to mascguy:
Replying to lukaso:
The solution is probably to revert libgit2 in macports. Clearly the build is not pulling the embedded version when it is linking (or so it seems).
No, that's probably not going to happen, unless this is more wide-spread. And given that other Rust-based ports build fine with the latest release of
libgit2
- includingcargo
andrust
itself - that strongly suggests this is arav1e
-specific issue.
Here's the thing to revert: [c00e8c728a0458fcf0e1d2c555126d6bb642feaa/macports-ports] and here's the PR in libgit2
which is labeled as sys api breaking
: https://github.com/libgit2/libgit2/pull/6563.
This port has been broken for over a month now for one reason or another. It was also last broken in december 2022. Maybe it's time to cut rav1e
loose? I tried to remove the dependency in the GIMP build by excluding it from libheif
but it appears to be included in some other port that somehow I am including so that didn't help.
comment:13 Changed 12 months ago by mascguy (Christopher Nielsen)
Replying to lukaso:
This port has been broken for over a month now for one reason or another. It was also last broken in december 2022. Maybe it's time to cut
rav1e
loose? I tried to remove the dependency in the GIMP build by excluding it fromlibheif
but it appears to be included in some other port that somehow I am including so that didn't help.
Okay, just tried building rav1e
from source. And sure enough, our port is currently broken.
But let me stress that's it's very likely our port that's broken, not upstream.
Why do I say that? Well, take a look at the portfile for rav1e
, and specifically the crate for libgit2-sys
:
libgit2-sys 0.14.2+1.5.1
That crate corresponds to libgit2
1.5.1. Whereas git2-rs
upstream has released later releases, including the latest: libgit2-sys-0.16.1+1.7.1
. And the latter corresponds to the latest libgit2
release. (Per the +1.7.1
suffix.) Which is what we should probably be used. Though perhaps one of the previous releases, corresponding to libgit2
1.6.x, might work too. Dunno without more research/testing.
In any case, updating that crate, likely involves updating others as well. Perhaps that's automated via a MacPorts script somewhere? (Not a Rust guru here, otherwise I would have fixed this by now.)
So I'm quite certain this is 100% our issue.
As for downgrading libgit2
: Simply reverting one commit isn't going to cut it. We'd also need to downgrade py-pygit2
, and a number of others. And then rebuild every dependent - including rust
and cargo
, along with numerous other dependents. (As the downgrade would break the ABI.) So no, not going to happen, when it seems very clear the rav1e
port is the one that needs to be fixed.
comment:14 follow-up: 15 Changed 12 months ago by mascguy (Christopher Nielsen)
Relative to eliminating rav1e
from the dependency equation: Most of that's in-place already.
The ports that depend on it, are the following:
$ gfind . -type f -name "Portfile" -print0 | gxargs -0 -r ggrep -E --regexp=":rav1e" ./multimedia/av1an/Portfile: port:rav1e \ ./multimedia/ffmpeg/Portfile: port:rav1e ./multimedia/ffmpeg-devel/Portfile: port:rav1e ./multimedia/ffmpeg6/Portfile: port:rav1e ./multimedia/libavif/Portfile: port:rav1e \ ./multimedia/libheif/Portfile: port:rav1e \
You can ignore ffmpeg-devel
and ffmpeg6
though, as you'll be using ffmpeg
.
Regardless, all but two of those support variants that can eliminate rav1e
from the equation:
- For
ffmpeg
, enable variant+slim
. - For
av1an
- assuming that's being pulled in for GIMP? - install with-av1
.
As for libavif
and libheif
, I maintain both of those. And I'm happy to provide something similar there, if needed. Though it's trivial to overlay your own versions of those two, dropping rav1e
from the deps:
- For
libavif
, remove the dep, and update-DAVIF_CODEC_RAV1E:BOOL=ON
to-DAVIF_CODEC_RAV1E:BOOL=OFF
. - Likewise, for
libheif
, remove the dep, and add configure argument-DWITH_RAV1E:BOOL=OFF
.
comment:15 follow-up: 16 Changed 12 months ago by lukaso (Lukas Oberhuber)
Regardless, all but two of those support variants that can eliminate
rav1e
from the equation:
- For
ffmpeg
, enable variant+slim
.- For
av1an
- assuming that's being pulled in for GIMP? - install with-av1
.As for
libavif
andlibheif
, I maintain both of those. And I'm happy to provide something similar there, if needed. Though it's trivial to overlay your own versions of those two, droppingrav1e
from the deps:
- For
libavif
, remove the dep, and update-DAVIF_CODEC_RAV1E:BOOL=ON
to-DAVIF_CODEC_RAV1E:BOOL=OFF
.- Likewise, for
libheif
, remove the dep, and add configure argument-DWITH_RAV1E:BOOL=OFF
.
I've done the overlay now for libheif
and libavif
. Should have done this ages ago! I think it would be worthwhile creating slim or removerav1e
versions for all of them since this appears to be a persistent issue. But I can move forward for now.
On an aside: It would be great to have much simpler tools for determining dependency...I'm always struggling to find what the dependencies are. Especially the dependencies all the way down.
comment:16 Changed 12 months ago by mascguy (Christopher Nielsen)
Replying to lukaso:
I've done the overlay now for
libheif
andlibavif
. Should have done this ages ago! I think it would be worthwhile creating slim orremoverav1e
versions for all of them since this appears to be a persistent issue. But I can move forward for now.
Sounds good, will try to tackle this over the next day or two.
On an aside: It would be great to have much simpler tools for determining dependency...I'm always struggling to find what the dependencies are. Especially the dependencies all the way down.
There definitely is, and macports provides a rich set of options to formally query for those. Take a look at man port
, specifically the pseudo-portnames.
For example:
$ port info --index --line --name depends:rav1e ffmpeg ffmpeg-devel ffmpeg6 libavif libheif av1an
Note: Option --index
is optional, but it speeds things up tremendously. (Simply because it utilizes the portindex for queries.)
Along with various other ways to do it, such as port echo
:
$ port echo depends:rav1e ffmpeg ffmpeg-devel ffmpeg6 libavif libheif av1an
But take a thorough read through man port
, for the various details. Cheers!
comment:17 Changed 12 months ago by ryandesign (Ryan Carsten Schmidt)
Description: | modified (diff) |
---|
Has multiple builds in it, so just focus on last one (they appear to be all the same, which might mean that macports tried to run it multiple times)