Opened 14 months ago
Last modified 10 months ago
#68178 assigned defect
rav1e: build fails due to git_allocator missing members
Reported by: | lukaso (Lukas Oberhuber) | Owned by: | mascguy (Christopher Nielsen) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | 2.8.1 |
Keywords: | Cc: | MarcusCalhoun-Lopez (Marcus Calhoun-Lopez), Dave-Allured (Dave Allured), ddrum2000, Iritscen, pspanja (Petar Španja), ahrex, hnarayanan (Harish Narayanan), sudheerhebbale (Sudheer Hebbale), phloxic (Christian Ebert) | |
Port: | rav1e |
Description (last modified by ryandesign (Ryan Carsten Schmidt))
First reported on #68026, via @macsguy:
This may require updating the version of the git2 crate [used by rav1e], now that libgit2 has been updated to 1.7.1:
libgit2/src/util/array.h:59:19: error: no member named 'greallocarray' in 'git_allocator' if ((new_array = git__reallocarray(a->ptr, new_size, item_size)) == NULL) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libgit2/src/libgit2/annotated_commit.c:34:21: error: no member named 'gcalloc' in 'git_allocator' annotated_commit = git__calloc(1, sizeof(git_annotated_commit)); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libgit2/src/libgit2/annotated_commit.c:48:34: error: no member named 'gstrdup' in 'git_allocator' annotated_commit->description = git__strdup(description); ^~~~~~~~~~~~~~~~~~~~~~~~
I gave a shot at trying to update the crates, but got errors after updating the crate list:
:info:extract error: failed to select a version for the requirement `git2 = "^0.15"` :info:extract candidate versions found which didn't match: 0.18.0 :info:extract location searched: directory source `/Users/lukasoberhuber/macports-gimp3-arm64/var/macports/build/_Users_lukasoberhuber_project_ports_multimedia_rav1e/rav1e/work/.home/.cargo/macports` (which is replacing registry `crates-io`) :info:extract required by package `built v0.5.2` :info:extract ... which satisfies dependency `built = "^0.5.2"` of package `rav1e v0.6.6 (/Users/lukasoberhuber/macports-gimp3-arm64/var/macports/build/_Users_lukasoberhuber_project_ports_multimedia_rav1e/rav1e/work/rav1e-0.6.6)` :info:extract perhaps a crate was updated and forgotten to be re-vendored? :info:extract As a reminder, you're using offline mode (--offline) which can sometimes cause surprising resolution failures, if this error is too confusing you may wish to retry without the offline flag. :info:extract Command failed: /Users/lukasoberhuber/macports-gimp3-arm64/bin/cargo --offline update :info:extract Exit code: 101
Attachments (3)
Change History (38)
Changed 14 months ago by lukaso (Lukas Oberhuber)
Attachment: | rav1e_main.log added |
---|
comment:1 Changed 14 months ago by mascguy (Christopher Nielsen)
Cc: | MarcusCalhoun-Lopez added |
---|---|
Description: | modified (diff) |
Owner: | set to mascguy |
Status: | new → assigned |
Summary: | rav1e fails to build on arm64 → rav1e: build fails on arm, due to git_allocator missing members; git2 crate [used by rav1e] may need to be updated |
Version: | → 2.8.1 |
comment:2 Changed 14 months ago by mascguy (Christopher Nielsen)
Keywords: | arm64 added |
---|
comment:3 Changed 14 months ago by rottegift (Sean Doran)
FWIW, I just built successfully with
port -v install rav1e configure.ldflags="-ld_classic -force_load /usr/lib/libiconv.dylib" configure.cflags="-Wl,-ld_classic"
which is ugly, but works up to not-yet-found problems with differing header files
otool -L /opt/local/bin/rav1e /opt/local/bin/rav1e: /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /opt/local/lib/libiconv.2.dylib (compatibility version 9.0.0, current version 9.1.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.0.0)
the -force_load
probably is portable to older macOS/xcode.
I don't think this is the right solution, but if one was, say, in the middle of building something with rav1e as a dependency, this might get the job done.
comment:4 Changed 14 months ago by mascguy (Christopher Nielsen)
Lukas, my thoughts regarding updating the version of the git2 crate may be off in left field, given the following:
The source for libgit2 is included in the libgit2-sys crate so there's no need to pre-install the libgit2 library, the libgit2-sys crate will figure that and/or build that for you.
Given that - combined with the fact that we haven't changed the rav1e
port - we should be fine as-is.
Can you try a clean build of rav1e
, without any prior history/log, to ensure the same results occur?
comment:5 Changed 14 months ago by Dave-Allured (Dave Allured)
Cc: | Dave-Allured added |
---|
comment:6 Changed 14 months ago by lukaso (Lukas Oberhuber)
I did a forced uninstall and a port clean
. If I should do any more clean than that, let me know how.
Attaching the new log.
Changed 14 months ago by lukaso (Lukas Oberhuber)
Attachment: | rav1e_main.2.log added |
---|
Post force uninstall and clean of rav1e
comment:7 Changed 14 months ago by lukaso (Lukas Oberhuber)
OK, this actually happens on x86_64 as well. Attaching the log. Same problem.
Changed 14 months ago by lukaso (Lukas Oberhuber)
Attachment: | rav1e_x86_64_main.log added |
---|
comment:8 Changed 14 months ago by mascguy (Christopher Nielsen)
Summary: | rav1e: build fails on arm, due to git_allocator missing members; git2 crate [used by rav1e] may need to be updated → rav1e: build fails due to git_allocator missing members |
---|
comment:9 Changed 14 months ago by mascguy (Christopher Nielsen)
Keywords: | arm64 removed |
---|
comment:10 follow-up: 11 Changed 14 months ago by lukaso (Lukas Oberhuber)
So it looks like these missing functions were removed in v1.7.1. So that's why we are stuffed. Also meaning they are not using semantic versioning (boo on them). https://github.com/libgit2/libgit2/commit/9dd1bfe81c3aba6ce35ed85bf62fa75d9c61e1a6.
The solution is probably to revert libgit2 in macports. Clearly the build is not pulling the embedded version when it is linking (or so it seems).
comment:11 follow-up: 12 Changed 14 months ago by mascguy (Christopher Nielsen)
Replying to lukaso:
The solution is probably to revert libgit2 in macports. Clearly the build is not pulling the embedded version when it is linking (or so it seems).
No, that's probably not going to happen, unless this is more wide-spread. And given that other Rust-based ports build fine with the latest release of libgit2
- including cargo
and rust
itself - that strongly suggests this is a rav1e
-specific issue.
comment:12 follow-up: 13 Changed 14 months ago by lukaso (Lukas Oberhuber)
Replying to mascguy:
Replying to lukaso:
The solution is probably to revert libgit2 in macports. Clearly the build is not pulling the embedded version when it is linking (or so it seems).
No, that's probably not going to happen, unless this is more wide-spread. And given that other Rust-based ports build fine with the latest release of
libgit2
- includingcargo
andrust
itself - that strongly suggests this is arav1e
-specific issue.
Here's the thing to revert: [c00e8c728a0458fcf0e1d2c555126d6bb642feaa/macports-ports] and here's the PR in libgit2
which is labeled as sys api breaking
: https://github.com/libgit2/libgit2/pull/6563.
This port has been broken for over a month now for one reason or another. It was also last broken in december 2022. Maybe it's time to cut rav1e
loose? I tried to remove the dependency in the GIMP build by excluding it from libheif
but it appears to be included in some other port that somehow I am including so that didn't help.
comment:13 Changed 14 months ago by mascguy (Christopher Nielsen)
Replying to lukaso:
This port has been broken for over a month now for one reason or another. It was also last broken in december 2022. Maybe it's time to cut
rav1e
loose? I tried to remove the dependency in the GIMP build by excluding it fromlibheif
but it appears to be included in some other port that somehow I am including so that didn't help.
Okay, just tried building rav1e
from source. And sure enough, our port is currently broken.
But let me stress that's it's very likely our port that's broken, not upstream.
Why do I say that? Well, take a look at the portfile for rav1e
, and specifically the crate for libgit2-sys
:
libgit2-sys 0.14.2+1.5.1
That crate corresponds to libgit2
1.5.1. Whereas git2-rs
upstream has released later releases, including the latest: libgit2-sys-0.16.1+1.7.1
. And the latter corresponds to the latest libgit2
release. (Per the +1.7.1
suffix.) Which is what we should probably be used. Though perhaps one of the previous releases, corresponding to libgit2
1.6.x, might work too. Dunno without more research/testing.
In any case, updating that crate, likely involves updating others as well. Perhaps that's automated via a MacPorts script somewhere? (Not a Rust guru here, otherwise I would have fixed this by now.)
So I'm quite certain this is 100% our issue.
As for downgrading libgit2
: Simply reverting one commit isn't going to cut it. We'd also need to downgrade py-pygit2
, and a number of others. And then rebuild every dependent - including rust
and cargo
, along with numerous other dependents. (As the downgrade would break the ABI.) So no, not going to happen, when it seems very clear the rav1e
port is the one that needs to be fixed.
comment:14 follow-up: 15 Changed 14 months ago by mascguy (Christopher Nielsen)
Relative to eliminating rav1e
from the dependency equation: Most of that's in-place already.
The ports that depend on it, are the following:
$ gfind . -type f -name "Portfile" -print0 | gxargs -0 -r ggrep -E --regexp=":rav1e" ./multimedia/av1an/Portfile: port:rav1e \ ./multimedia/ffmpeg/Portfile: port:rav1e ./multimedia/ffmpeg-devel/Portfile: port:rav1e ./multimedia/ffmpeg6/Portfile: port:rav1e ./multimedia/libavif/Portfile: port:rav1e \ ./multimedia/libheif/Portfile: port:rav1e \
You can ignore ffmpeg-devel
and ffmpeg6
though, as you'll be using ffmpeg
.
Regardless, all but two of those support variants that can eliminate rav1e
from the equation:
- For
ffmpeg
, enable variant+slim
. - For
av1an
- assuming that's being pulled in for GIMP? - install with-av1
.
As for libavif
and libheif
, I maintain both of those. And I'm happy to provide something similar there, if needed. Though it's trivial to overlay your own versions of those two, dropping rav1e
from the deps:
- For
libavif
, remove the dep, and update-DAVIF_CODEC_RAV1E:BOOL=ON
to-DAVIF_CODEC_RAV1E:BOOL=OFF
. - Likewise, for
libheif
, remove the dep, and add configure argument-DWITH_RAV1E:BOOL=OFF
.
comment:15 follow-up: 16 Changed 14 months ago by lukaso (Lukas Oberhuber)
Regardless, all but two of those support variants that can eliminate
rav1e
from the equation:
- For
ffmpeg
, enable variant+slim
.- For
av1an
- assuming that's being pulled in for GIMP? - install with-av1
.As for
libavif
andlibheif
, I maintain both of those. And I'm happy to provide something similar there, if needed. Though it's trivial to overlay your own versions of those two, droppingrav1e
from the deps:
- For
libavif
, remove the dep, and update-DAVIF_CODEC_RAV1E:BOOL=ON
to-DAVIF_CODEC_RAV1E:BOOL=OFF
.- Likewise, for
libheif
, remove the dep, and add configure argument-DWITH_RAV1E:BOOL=OFF
.
I've done the overlay now for libheif
and libavif
. Should have done this ages ago! I think it would be worthwhile creating slim or removerav1e
versions for all of them since this appears to be a persistent issue. But I can move forward for now.
On an aside: It would be great to have much simpler tools for determining dependency...I'm always struggling to find what the dependencies are. Especially the dependencies all the way down.
comment:16 Changed 14 months ago by mascguy (Christopher Nielsen)
Replying to lukaso:
I've done the overlay now for
libheif
andlibavif
. Should have done this ages ago! I think it would be worthwhile creating slim orremoverav1e
versions for all of them since this appears to be a persistent issue. But I can move forward for now.
Sounds good, will try to tackle this over the next day or two.
On an aside: It would be great to have much simpler tools for determining dependency...I'm always struggling to find what the dependencies are. Especially the dependencies all the way down.
There definitely is, and macports provides a rich set of options to formally query for those. Take a look at man port
, specifically the pseudo-portnames.
For example:
$ port info --index --line --name depends:rav1e ffmpeg ffmpeg-devel ffmpeg6 libavif libheif av1an
Note: Option --index
is optional, but it speeds things up tremendously. (Simply because it utilizes the portindex for queries.)
Along with various other ways to do it, such as port echo
:
$ port echo depends:rav1e ffmpeg ffmpeg-devel ffmpeg6 libavif libheif av1an
But take a thorough read through man port
, for the various details. Cheers!
comment:17 Changed 14 months ago by ryandesign (Ryan Carsten Schmidt)
Description: | modified (diff) |
---|
comment:18 follow-up: 19 Changed 14 months ago by mascguy (Christopher Nielsen)
Lukas, I'm making progress, relative to updating rav1e
to use the latest version of libgit2-sys
, etc. (It simply required some time to review how dependencies are handled within the Rust/Cargo space.)
Presently dealing with compilation failures related to libssh2
, but will figure it out with enough time.
So hopefully this port will be fixed in the next few days, and you won't need to remove it from the mix long-term. Stay tuned...
comment:19 Changed 14 months ago by lukaso (Lukas Oberhuber)
Replying to mascguy:
Lukas, I'm making progress, relative to updating
rav1e
to use the latest version oflibgit2-sys
, etc. (It simply required some time to review how dependencies are handled within the Rust/Cargo space.)Presently dealing with compilation failures related to
libssh2
, but will figure it out with enough time.So hopefully this port will be fixed in the next few days, and you won't need to remove it from the mix long-term. Stay tuned...
Great. Thanks.
comment:20 Changed 14 months ago by Christopher Nielsen <mascguy@…>
comment:21 Changed 14 months ago by Christopher Nielsen <mascguy@…>
comment:22 Changed 14 months ago by lukaso (Lukas Oberhuber)
Thanks! Trying out the rav1e free ports now.
comment:24 Changed 14 months ago by mascguy (Christopher Nielsen)
Observers: The work-in-progress fixes for rav1e
- which may take some time to figure out - are being done via PR:
comment:25 Changed 14 months ago by lukaso (Lukas Oberhuber)
Confirming that rav1e
free ports has worked on my builds.
comment:26 Changed 14 months ago by ddrum2000
Cc: | ddrum2000 added |
---|
comment:27 Changed 13 months ago by Iritscen
Cc: | Iritscen added |
---|
comment:28 Changed 13 months ago by mascguy (Christopher Nielsen)
Cc: | pspanja ahrex added |
---|
Has duplicate issue:68272
comment:29 Changed 13 months ago by mascguy (Christopher Nielsen)
For new folks just added to this ticket: Ports which previously depended on rav1e
, no longer do so by default:
ffmpeg
ffmpeg-devel
ffmpeg6
libavif
libheif
The only port which still does, is av1an
. And the latter is a standalone port, which isn't pulled in by anything.
So, what's the takeaway? Unless you have a definite need for rav1e
, it's no longer in play by default. And it can be uninstalled, eliminating the errors.
Ultimately we do hope to fix it eventually, and we have a work-in-progress PR for that:
But at least for the near term, you needn't worry about rav1e
any longer. (Again, unless you have a definite need for it.)
comment:30 Changed 13 months ago by Iritscen
I see, thanks for explaining. I uninstalled rav1e, and nothing seems to miss it :-)
comment:31 Changed 13 months ago by hnarayanan (Harish Narayanan)
Cc: | hnarayanan added |
---|
comment:32 Changed 12 months ago by ryandesign (Ryan Carsten Schmidt)
Cc: | sudheerhebbale added |
---|
Has duplicate #68696.
comment:33 follow-up: 34 Changed 11 months ago by kencu (Ken)
comment:34 Changed 11 months ago by mascguy (Christopher Nielsen)
Replying to kencu:
Has duplicate #68948 and the solution in #68948 that worked for the OP was to deactivate libgit2.
Bear in mind that rav1e
is no longer in play by default, per comment:29.
Still, glad to hear that users have an easy workaround!
comment:35 Changed 10 months ago by phloxic (Christian Ebert)
Cc: | phloxic added |
---|
Has multiple builds in it, so just focus on last one (they appear to be all the same, which might mean that macports tried to run it multiple times)