webkit2-gtk @2.26.2_0+minibrowser+x11 GIR Files Malformed

[jmercouris (John Mercouris)]

Looking at /opt/local/share/gir-1.0/WebKit2-4.0.gir:

... 
  <namespace name="WebKit2"
             version="4.0"
             shared-library="@rpath/libwebkit2gtk-4.0.37.dylib,@rpath/libjavascriptcoregtk-4.0.18.dylib"
             c:identifier-prefixes="WebKit"
             c:symbol-prefixes="webkit">
...

As you can see, there are @rpath references. This is causing problems when trying to use gobject introspection. I get the following error: Could not locate webkit_web_view_new: dlopen(@rpath/libjavascriptcoregtk-4.0.18.dylib, 9): image not found

I looked at another GIR file for a known working one: /opt/local/share/gir-1.0/Gtk-3.0.gir, that one looks like this:

... 
  <namespace name="Gtk"
             version="3.0"
             shared-library="/opt/local/lib/libgtk-3.0.dylib,/opt/local/lib/libgdk-3.0.dylib"
             c:identifier-prefixes="Gtk"
...

As you can see, no @rpath. This leads me to believe that there is a problem with the port. Also please note at the top of the GIR file it says:

<!-- This file was automatically generated from C sources - DO NOT EDIT!
To affect the contents of this file, edit the original C definitions,
and/or use gtk-doc annotations.  -->

So, one cannot simply edit the file. They have to edit the file that generates these files. Any insight in solving these problems to properly generate these files would be greatly appreciated. Thank you.

[kencu (Ken)]

The libraries themselves don't have @rpath references, just these gir files, it appears:

$ otool -L /opt/local/lib/libjavascriptcoregtk-4.0.18.dylib
/opt/local/lib/libjavascriptcoregtk-4.0.18.dylib:
	/opt/local/lib/libjavascriptcoregtk-4.0.18.dylib (compatibility version 18.0.0, current version 18.15.2)
	/opt/local/lib/libglib-2.0.0.dylib (compatibility version 5801.0.0, current version 5801.3.0)
	/opt/local/lib/libicudata.65.dylib (compatibility version 65.0.0, current version 65.1.0)
	/opt/local/lib/libicui18n.65.dylib (compatibility version 65.0.0, current version 65.1.0)
	/opt/local/lib/libicuuc.65.dylib (compatibility version 65.0.0, current version 65.1.0)
	/opt/local/lib/libgio-2.0.0.dylib (compatibility version 5801.0.0, current version 5801.3.0)
	/opt/local/lib/libgobject-2.0.0.dylib (compatibility version 5801.0.0, current version 5801.3.0)
	/opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 5.0.1)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.11)
	/usr/lib/libc++abi.dylib (compatibility version 1.0.0, current version 5.0.1)

Dave -- I don't know much about gobject-introspection -- should this be fixed? (Presume so). And if so, any trouble with just Bogarting the ${prefix} in instead of @rpath with a reinplace in a post-destroot correction?

If you know how to change that properly during the (exceeding complex 2.5 hour) build of webkit2-gtk, then naturally I'm all ears!

[jmercouris (John Mercouris)]

I did receive the following helpful message from a colleague:

.gir files are compiled, see ​https://gi.readthedocs.io/en/latest/

More precisely, it's possible that the macports did not _re-scan_ the source.

Perhaps that may be of some help.

[kencu (Ken)]

Interesting link. Perhaps Dave will chime in.

For now, I would think if you just hand edit the three gir files and change @rpath to your prefix (usually /opt/local) you should be good to go with whatever it is you're up to while he sorts this out.

I suspect that is exactly what we will wind up doing in that post-destroot phase, unless something wonderful comes forth from poring over the cmake spaghetti.

[jmercouris (John Mercouris)]

I did try editing one of the gir files, and I still had issues, perhaps I need to edit *all* of them, I will give that a try tomorrow, thanks for the tip and for looking into this!

[kencu (Ken)]

or maybe they have been compiled already and that approach isn't going to work...

[jmercouris (John Mercouris)]

Here is something very fascinating:

/opt/local/share/gobject-introspection-1.0/Makefile.introspection

Here is what the file says:

# * Input variables:
#
#   INTROSPECTION_GIRS - List of GIRS that should be generated
#   INTROSPECTION_SCANNER - Command to invoke scanner, normally set by
#      GOBJECT_INTROSPECTION_REQUIRE/CHECK() in introspection.m4
#   INTROSPECTION_SCANNER_ARGS - Additional args to pass in to the scanner
#   INTROSPECTION_SCANNER_ENV - Environment variables to set before running
#      the scanner
#   INTROSPECTION_COMPILER - Command to invoke compiler, normally set by
#      GOBJECT_INTROSPECTION_REQUIRE/CHECK() in introspection.m4
#   INTROSPECTION_COMPILER_ARGS - Additional args to pass in to the compiler
#
# * Simple tutorial
#
# Add this to configure.ac:
#   -Wno-portability to AM_INIT_AUTOMAKE
#   GOBJECT_INTROSPECTION_CHECK([0.6.7])
#
# Add this to Makefile.am where your library/program is built:
#   include $(INTROSPECTION_MAKEFILE)
#   INTROSPECTION_GIRS = YourLib-1.0.gir
#   YourLib-1.0.gir: libyourlib.la
#   YourLib_1_0_gir_NAMESPACE = YourLib
#   YourLib_1_0_gir_VERSION = 1.0
#   YourLib_1_0_gir_LIBS = libyourlib.la
#   YourLib_1_0_gir_FILES = $(libyourlib_1_0_SOURCES)
#   girdir = $(datadir)/gir-1.0
#   dist_gir_DATA = YourLib-1.0.gir
#   typelibdir = $(libdir)/girepository-1.0
#   typelib_DATA = YourLib-1.0.typelib
#   CLEANFILES = $(dist_gir_DATA) $(typelib_DATA)
#

...

As you can see it is a little tutorial. Maybe it is just some things in the makefile that need to be tweaked (they are tweaked for GTK)?

[jmercouris (John Mercouris)]

To answer the other question, editing the gir files in place and removing @rpath unfortunately did not seem to work.

[jmercouris (John Mercouris)]

More information. Editing the GIR files in place *does* work if you then recompile and regenerate the type lib files (type lib files are like compiled gir files). Here is what I did:

1. Installed gobject-introspection port which gave me access to the g-ir-compiler

2. Edit the GIR files located in

/opt/local/share/gir-1.0/WebKit2WebExtension-4.0.gir to remove references to @rpath and replace them with /opt/local/lib, so now the shared-library line looks like this:

shared-library="/opt/local/lib/libwebkit2gtk-4.0.37.dylib,/opt/local/lib/libjavascriptcoregtk-4.0.18.dylib"

3. Used the compiler gir compiler:

   g-ir-compiler WebKit2WebExtension-4.0.gir > /opt/local/lib/girepository-1.0/WebKit2WebExtension-4.0.typelib
   

4. Changed owner and group on new type lib file to match other typelibs

5. After having new typelibs generated from my hand modified GIR files I can successfully launch a test web view, however, it does not work. It just shows a blank view. How to debug this?

Screenshot: ​https://i.imgur.com/u1mnV3R.png

[kencu (Ken)]

glad you're making progress. recompiling after editing makes sense. we could likely do that in a post-destroot block too.

seems to me this issue might affect all ports that use cmake and introspection.

will see about cmake's "build with install rpath on/off" setting, or some way of making it do the right thing here.

no idea about your black screen. thry installing "surf" or running the MiniBrowser and see if those work for you.

[jmercouris (John Mercouris)]

I would like to try running the MiniBrowser, can you please tell me how to do so? Thank you.

[kencu (Ken)]

port contents webkit2-gtk | grep Mini

will find it for you.

[jmercouris (John Mercouris)]

MiniBrowser AND "surf" work for me, I'm all out of ideas. I guess I will look deeper into GIR.

[kencu (Ken)]

g-ir-scanner seems to pull out the library name in this file /opt/local/lib/gobject-introspection/giscanner/shlibs.py and specifically in this part:

        platform_system = platform.system()
        if platform_system == 'Darwin':
            args.extend(['otool', '-L', binary.args[0]])

which looks like it just asks the library for it's name, and uses that.

I guess at the time this scanning is done, that library must have an @rpath name, even though cmake is set up with this:

-DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON -DCMAKE_INSTALL_RPATH="/opt/local/lib"

so -- next step would be, I suppose, to build webkit2-gtk, and before destrooting or installing it, go look at the libraries of interest and see how they are named at that point in time.

If they do have @rpath names in the build tree, and I guess they will, then the next step would be to figure why and how that might be managed.

[jmercouris (John Mercouris)]

Is there a way to partially build a port? I could help with the testing of this if you like. Just tell me what to do and I can run the tedious process on my machine :-)

[kencu (Ken)]

sure, easy, it all happens in stages.

sudo port -v fetch
sudo port -v checksum
sudo port -v extract
sudo port -v patch
sudo port -v configure
sudo port -v build
sudo port -v destroot
sudo port -v install

are all separate stages, that run the ones above them. So sudo port -v build will run fetch, checksum, extract, patch, configure, and then build.

To find the working directory, you run

port work webkit2-gtk

or I usually just

cd `port work webkit2-gtk`

and then look around in there for the source and build folders, etc.

[jmercouris (John Mercouris)]

OK, thank you I will investigate and report if I find anything.

[kencu (Ken)]

just for my education, all of the ports on macports that use webkit2-gtk seem to be doing so without any troubles. How does this g-ir-scanner issue show up and affect you?

[jmercouris (John Mercouris)]

Webkit2-gtk works just fine when linking against the library, which is why all other ports don't have any issue. The problem with malformed GIR files is that one cannot use g-object-introspection based bindings for FFI with other programming languages.

In my case I am trying to use cl-gobject-introspection (​https://github.com/andy128k/cl-gobject-introspection) in order to create an FFI port for the browser I am writing, Next (​https://github.com/atlas-engineer/next). However, if I can display every other widget except WebkitGTK+ widgets, then I have a bit of a problem :-)

[kencu (Ken)]

Thanks for that. We certainly want to help make this work for you, and indeed there may be an issue with the way webkit2-gtk is built on MacPorts (where we don't use the rpath in the end, on purpose). Hope we can solve it for you.

[kencu (Ken)]

This issue looks like it was worked on and fixed in various places, eg <​https://github.com/NixOS/nixpkgs/pull/46310/commits/363f78fb4c3e7dc777d4870ea5ce343c9fc4ba40> and <​https://gitlab.gnome.org/GNOME/gobject-introspection/issues/222>

[jmercouris (John Mercouris)]

Interesting, it seems to have been merged into the gobject-introspection code. How come we still have this error? Are we using an older version of the python lib?

[kencu (Ken)]

Yes, it appears we are using an older version of gobject-introspection that does not have the @rpath fix in it:

$ port livecheck gobject-introspection
gobject-introspection seems to have been updated (port version: 1.58.3, new version: 1.62.0)

so -- that's what we need to do, then -- update gobject-introspection. Usually there is a reason why Dave hasn't yet done this, so we'll see what's going on -- perhaps it's just not hit the top of his to-do list yet.

[kencu (Ken)]

and -- oh, the horror -- EVERY port that uses gobject-introspection will need to be rev-bumped... the buildbots are crying already ....

[jmercouris (John Mercouris)]

I can confirm that the changes to the type lib code as I described earlier do work. The rendering issue in my main project is a product of my g-object-library and/or the code I am using to generate the web view. I know this because I was able to successfully use the g-object web view bindings in Python to generate a test web view.

[kencu (Ken)]

There is already a PR in play to update gobject-introspection to the current version, I note. It's been under review for a while now, so hopefully will move along <​https://github.com/macports/macports-ports/pull/5759> soon.

[michaelld (Michael Dickens)]

Please try out this PR < ​https://github.com/macports/macports-ports/pull/6544 > and see if it solves the issue. It does for me. It's an interim step between the current GI version and the one from <​​https://github.com/macports/macports-ports/pull/5759> ... which isn't going anywhere quickly it looks like & is already outdated (1.64.0 is out; the PR is for 1.62.0).

[kencu (Ken)]

​https://github.com/macports/macports-ports/pull/6544/commits/24462b33afc89baf4a054148d8f9bfed96bf513a