KF5: PortGroup and initial ports (frameworks)

[RJVB (René Bertin)]

Here's an initial implementation of a PortGroup and the basic ports required to deploy KF5 through MacPorts, thanks in large part to lots of preliminary heavy lifting behind the scenes by Marko Käning. Without him this would have been much more work.

This initial submission contains the PortGroup, the Frameworks, and a few tightly related ports. As always, further development can be followed via ​https://github.com/RJVB/macstrop/tree/master/kf5 .

KDE5 is not a straightforward thing to deploy (package), in some ways even less so than KDE4, but in other ways "things" have improved sufficiently that a lot can be abstracted and automatised (more so than with KDE4).

Most of that is done in the PortGroup, which contains the basic logic that makes declaring a port for a given KF5 application relatively easy. It also contains version definitions, because despite being individual packages, KF5 components are released in synchronised fashion, and won't build against older components. The PortGroup is also responsible for including the Qt5 PortGroup, and does so by indicating a preference for port:qt5-kde.

The frameworks themselves, the building blocks on which KF5 applications are based (and "interesting/useful extensions to Qt5") are a bit different. I have chosen to bundle all frameworks as subports in a single (huge) portfile, in order to increase sharing of patches and reinplace statements (something the benefits of which I hope I won't have to explain). The meta-port is almost complete; it lacks only a number of "porting aid" frameworks which I plan to add when needed (or "in due time").

That meta-port, KF5-Frameworks, also contains the variant that I consider crucial for MacPorts, the one that activates the XDG-compliant mode of Qt's QStandardPaths. In a nutshell, this is the mode in which makes Qt5 (and KF5) code look for shared resources under ${prefix}/share, and use ~/.config, ~/.local and ~/.cache instead of the OS X-ish equivalents. Individual applications (pure Qt5 or KF5) may be adapted or even conceived to use the Apple-equivalents of these locations, but KF5 applications as a whole are designed primarily to function in a Freedesktop (XDG) universe, and cohabit/interact with other XDG-compliant applications that are *not* based on Qt5 (e.g. KDE4!). I consider this something MacPorts cannot ignore or not support. I'd even go so far as to remove the possibility to build KF5 otherwise, but as long as the QSP patch is only shipped with port:qt5-kde I see no other way than to provide a variant that becomes a default variant if the QSP/XDG activator is detected at build time. The KF5 PortGroup indicates a preference for port:qt5-kde and therefore it is that port which will be installed during a fresh install or on a build bot. I am aware this is a potential violation of the repeatable build principle, but only for/by users who go against a recommended dependency (and these users will be obliged to install all KF5 ports from source).

NB: The XDG activator is an additional link module that I bundle with my port:qt5-kde; linking it with all KF5 frameworks means that any KF5 application will use QStandardPaths in XDG-compliant mode; this blunt approach is required since there is no single KF5 component with which all dependents are obliged to link. Making the compliancy mode optional (and inactive by default) means that "pure Qt5" applications function with native OS X QStandardPaths, and will thus behave exactly as their counterparts if installed from an official installer other external source. I consider that of equal importance as the repeatable build principle.

Also note that there are a number of tweaks that allow me to use the same build and packaging logic on Linux, for testing and comparison; those tend to relax dependencies so they are obtained from the host. I'm not adamant on "getting these in" of course, but as long as I'm principle maintainer on these files I'm also not very keen on removing them .

[RJVB (René Bertin)]

the KF5 PortGroup. It has a 1.1 version out of recognition for Marko's 1.0 version

[RJVB (René Bertin)]

Portfile for kf5-oxygen-icons

[RJVB (René Bertin)]

PortFile for the Breeze icons

[RJVB (René Bertin)]

Portfile for the kf5-gpgmepp "wannabe" framework

[mkae (Marko Käning)]

Hi René, this all looks quite complicated, but interesting as it allows for automatic dependency generation of devel ports. Very cool indeed! Am quite distracted with real life issues these days, which is why I can't promise testing all this very quickly, especially not both variants standard and XDG-compliant...

BTW, I am wondering whether one can get kate to build with this already!

[mkae (Marko Käning)]

This is an impressive amount of work!

Yet, how do you:

• manage version updates for all those sub-ports now? Do you use some script to get the checksums in sync with the corresponding version?
• create the individual lists of dependencies of all those sub-ports, is it manually done or scripted??

[RJVB (René Bertin)]

get kate to build with this already!

It's high on my list of new ports to add. I'm expecting two things: 1) it should build and 2) it won't be that easy ;)

manage version updates for all those sub-ports now? Do you use some script to get the checksums in sync with the corresponding version?

Frameworks are synced and thus always have the same version (possibly with some additional things added in case of devel ports). I have indeed considered putting all checksums into an array and then create a script that calls say port -v checksum for all subports in the correct order. I'm afraid the array won't improve readability though; what we'd really need is a tool that replaces the checksum values automatically. "base" already shows what they should be based on what the portfile provides.

create the individual lists of dependencies of all those sub-ports, is it manually done or scripted??

Done manually; I don't really see any other way to do it because not all frameworks declare their dependencies the exact same way.

For both maintenance chores I can only hope that updates will be less and less frequent, and that the dependencies won't change much.

[RJVB (René Bertin)]

So, thumbs-up:

I have an initial implementation of a port:kf5-kate, and it works. Glitches:

• the application is built without an icon
• I'll have to figure out what causes the repeated

  void QCocoaMenu::insertNative(QCocoaMenuItem *, QCocoaMenuItem *) Menu item "Clipboard &History" is already in menu "Edit" , remove it from the other menu first before inserting
  void QCocoaMenu::insertNative(QCocoaMenuItem *, QCocoaMenuItem *) Menu item "&Bookmarks" is already in menu "" , remove it from the other menu first before inserting
  virtual void QCocoaMenu::removeMenuItem(QPlatformMenuItem *) Item to remove does not belong to this menu
  virtual void QCocoaMenu::removeMenuItem(QPlatformMenuItem *) Item to remove does not belong to this menu
  

I hope that's just a question of a menuRole being set wrongly.

[RJVB (René Bertin)]

teaser: kate5 with the QtCurve theme

[mkae (Marko Käning)]

Replying to rjvbertin@…:

I have indeed considered putting all checksums into an array and then create a script that calls say port -v checksum for all subports in the correct order. I'm afraid the array won't improve readability though; what we'd really need is a tool that replaces the checksum values automatically. "base" already shows what they should be based on what the portfile provides.

Yes, such a replacement script is needed.

Done manually; I don't really see any other way to do it because not all frameworks declare their dependencies the exact same way.

Well, that's why I came up with my ​(admittedly quite awkward but functioning) install.sh script, which uses ​KDE's dependency metadata to create a valid dependency list for each and every framework or application.

[RJVB (René Bertin)]

Some more cleaning up and "backports" of patches from KDElibs4 and other "base" KDE4 ports.

Well, that's why I came up with my ​(admittedly quite awkward but functioning) install.sh script, which uses ​KDE's dependency metadata to create a valid dependency list for each and every framework or application.

If you used that for all your KF5 ports, it is not very reliable. I've had to edit more than one list of dependencies. The ultimate check will remain "manual", also to catch build dependencies like those imposed by KI18n. The translation files are shipped as source, and thus require gettext and a Python interpreter in order to be installed. KI18n provides a CMake macro to point CMake to the directory containing the translation files, and that macro adds the gettext and Python dependencies to the build. The thing is that several ports rely on CMake to pull in the KI18n dependency (via its handling of public link interfaces). The port:kf5-ki18n dependency will of course be added indirectly in that case, but AFAIK build dependencies are not inherited.

[mkae (Marko Käning)]

If you used that for all your KF5 ports, it is not very reliable. I've had to edit more than one list of dependencies.

Please, can you keep a record of which dependencies were missing in my first attempt to this endeavor!?! Perhaps some of them need to be reported upstream...

[RJVB (René Bertin)]

In a matter of speaking track is being kept: my and your ports both declare their dependencies, and one only needs to compare those lists.

It's quite likely that the method you used isn't incomplete in the sense that it leaves packages without the required dependencies (though the repository lags almost by definition of course). I haven't taken the time to evaluate to what extent framework dependencies resolve to the correct list when following inter-framework dependencies (if you see what I mean). I just prefer to match the dependency lists I declare to those that are printed in the cmake summary; and as I said, even those aren't always complete for MacPorts purposes, esp. not if you want to declare the right kind of dependencies (build, regular, runtime, etc).

[RJVB (René Bertin)]

KDE4 provided its translations via a separate mechanism which allowed separate ports to be created for each translation.

KF5 has been realigned with the usual all-inclusive approach, in which each package ([sub]port) installs every translation that has been made for it.

I don't like cluttering my disk with countless small and for the most part useless files, so I implemented a scheme that prunes ${prefix}/share/locale during the post-destroot phase, based on settings in ${prefix}/etc/macports/locales.tcl (.tcl because I wanted to keep this lightweight). If that file doesn't exist or doesn't define the expected list, nothing is done. Otherwise, all translations that don't match those listed in locales.tcl are deleted. The variant and post-destroot block are implemented in a PortGroup file.

No port should ever set this variant as a default variant, evidently.

[RJVB (René Bertin)]

KF5 Frameworks have been updated to 5.19.0 , and this time I've followed that update.

These new versions of the Portfile, PortGroup and patches also benefit from Marko's testing and patience.

[RJVB (René Bertin)]

NB: this also requires the latest Extra-Cmake-Modules, available ​here as long as someone hasn't pushed it to port:kde-extra-cmake-modules.

[mkae (Marko Käning)]

BTW, your change for Kate made the "Process Name" appear properly in OSX' Activity Monitor, but it is still not done properly for Kwrite!

[RJVB (René Bertin)]

I know; I'm under the impression that KWrite is a simplified editor and as such a "bit more devoid of interest" than Kate itself.

I've submitted the patch upstream; given that the main dev is rather reactive I hope that he'll take care of kwrite (and then I'll incorporate/cherry-pick the upstream patch until it appears in a release).

[mkae (Marko Käning)]

Replying to rjvbertin@…:

I know;...

Sorry, I saw only later that you were mentioning that already in your RR at KDE's RB. :)

[RJVB (René Bertin)]

A heads-up:

I'm in the process of updating to KF5 5.20.0, which will contain the 1st step in a transition to a different QSP/XDG approach.

Let me recapitulate.

Qt5 introduced a special class to obtain the standard locations for various resources, QStandardPaths (QSP). On OS X, it will return locations like Library/Application Support where on other Unices it will return XDG-compliant paths like share or local/share instead.

KF5 code uses that class, among others to find resources like icon themes but also DBus services (at least to register them). If KF5 applications are to function and interact with other XDG/FreeDesktop applications (that are not Qt5-based) as expected, QSP must be patched so it can return XDG-compliant locations.

My port:qt5-kde contains a patch that allows to toggle QSP into an XDG-compliant mode, and this is where a transition is taking place. The patch provides both an XDG activator via a linker module (.framework) and via a subclass (QExtStandardPaths; QESP) that can replace QSP via a preprocessor macro.

In the previous approach, I used a simple shortcut which defines "KF5 application" as an application that uses even a single KF5 framework. Each KF5 framework was linked with the activator module, causing QSP to be switched to XDG-compliant mode when the framework was loaded. That works fine, but can have side-effects for instance when loading a KF5 plugin into Qt's Designer.

I'm thus transitioning to the QESP-based approach, where code will use QExtStandardPaths instead of QStandardPaths when a preprocessor token is defined; QESP overrides the QSP functions so that they use either native or XDG-compliant locations depending on the value of another preprocessor token. These preprocessor tokens are set in the KF5 PortGroup, and thus the definition for "KF5 application" becomes "application that is built using [= part of] the KF5 PortGroup". Ditto for KF5 libraries and frameworks: they will use XDG-compliant locations internally (or native locations if the +nativeQSP variant is used), without altering the behaviour of "pure Qt" applications that use them.

During the transition period I will actually be using both approaches in order to be sure everything works as planned. Code will be built using QESP configured for XDG-compliant locations, but the KF5 frameworks will still be linked with the QSP/XDG activator module. That activator can be disabled via an env. variable, which allows to compare behaviour with and without, without need for relinking.

FWIW, I asked for feedback on this choice several weeks ago. I got none, so I'm now doing what I think best (and easiest to maintain).

[RJVB (René Bertin)]

This is a rather significant update as it does away completely with the +qspXDG variant (kept only as a stub) and now requires Qt 5.4.0 as a minimum.

I've included a new port in the tarball, port:kf5-osx-integration, which provides the KDE platform theme plugin that used to be part of port:kf5-frameworkintegration. Upstream removed it from there, and rather than using a patch that reinstates it, it is now a "KDE scratch" project which may end up being incorporated as the OS X equivalent of the plasma-integration package.

There's also an updated version of the QtCurve port.

[RJVB (René Bertin)]

kf5-kinit now installs a kdeinit5 launchd plist; starting it via kded5's launchd plist turned out not to be reliable. This also allows to start only kdeinit5 and not kded5 which isn't necessarily useful for everyone.

[RJVB (René Bertin)]

updated to 5.24.0

[mkae (Marko Käning)]

As shown in ​this KDE-MAC post I have been able to build kf5-Frameworks with almost 40 additional KF5-based ports.

So, I guess, we are nearing finally the release of port qt5-kde and all KF5-related ports to MacPorts?!

[mkae (Marko Käning)]

René, I think it might be a good idea to get in touch with ArchLinux user ​Mateusz Mikuła who is trying to get KF5 to install on Windows based on ​MSYS2?!

[mati865@…]

Cc Me!

[mati865@…]

Hello, I'm porting KDE to Windows using MSYS2. Right now I'm patching every usage of QStandardPaths::ConfigLocation or QStandardPaths::GenericDataLocation to use QLibraryInfo instead as MSYS2 don't want to diverge from upstream Qt. Although it works it would be better to have it fixed on the KDE or Qt side, like creating new CMake options or enum/class in the Qt. Right now I'll focus on getting KDE to work and improve it later after discussion with the others MSYS2 users.

[mati865@…]

On second though I'll create modified Qt like you did.

[mkae (Marko Käning)]

Replying to mati865@…:

On second though I'll create modified Qt like you did.

Hi mati865, how are things going for you on MSYS2 wrt a KF5'ish qt5 patch and all the KF5 frameworks? Can you post a status update for your endeavor here from time to time?

[RJVB (René Bertin)]

I've been following this from a distance while on holidays.

Sadly I've never really looked at how the MSWin QStandardPaths (QSP) class is implemented and what kind of paths it returns compared to what it "should" return for MSYS2. I've never really used MSYS2 but from what I remember it aims to do similar things as Cygwin but without or with much less overhead (and without a dedicated runtime DLL) (?).

That said, yes, I do think your best bet would be to patch Qt like we did. I must admit that my first reaction was that our patches will be so different that there's little to "collaborate" on. On second thought there might be a reason. It is my intention to try and get our patch into Qt. The chances to that should increase if other platforms can also benefit from a modified/extended QSP class that 1) allows applications to be build such that they use appropriate paths instead of the "native" ones while 2) the default way of building code still yields the default, "native" paths.

I should add that someone submitted a patch for Qt review that allows something like this for the writable paths, using the qt.conf file. If that sounds interesting but not familiar, please don't hesitate to remind me in a couple of days so I can dig up the link.

[mati865@…]

Replying to mk@…:

Replying to mati865@…:

On second though I'll create modified Qt like you did.

Hi mati865, how are things going for you on MSYS2 wrt a KF5'ish qt5 patch and all the KF5 frameworks? Can you post a status update for your endeavor here from time to time?

I was waiting with reply until I finish the patch. It took so long because recompilation of Qt with Core 2 Duo CPU takes it's time (well, a lot of it) but now the patch is almost done. While it's current state prevents it from getting merged into Qt it can still be used as starting point: ​https://github.com/Alexpux/MINGW-packages/pull/1640

Replying to rjvbertin@…:

I've been following this from a distance while on holidays.

Sadly I've never really looked at how the MSWin QStandardPaths (QSP) class is implemented and what kind of paths it returns compared to what it "should" return for MSYS2. I've never really used MSYS2 but from what I remember it aims to do similar things as Cygwin but without or with much less overhead (and without a dedicated runtime DLL) (?).

That said, yes, I do think your best bet would be to patch Qt like we did. I must admit that my first reaction was that our patches will be so different that there's little to "collaborate" on. On second thought there might be a reason. It is my intention to try and get our patch into Qt. The chances to that should increase if other platforms can also benefit from a modified/extended QSP class that 1) allows applications to be build such that they use appropriate paths instead of the "native" ones while 2) the default way of building code still yields the default, "native" paths.

I should add that someone submitted a patch for Qt review that allows something like this for the writable paths, using the qt.conf file. If that sounds interesting but not familiar, please don't hesitate to remind me in a couple of days so I can dig up the link.

qstandardpaths_win.cpp compared to qstandardpaths_unix.cpp is a unreadable because of Win API usage. Since MSYS2 use Linux like paths it was easy to implement new functions on top of qstandardpaths_unix.cpp with added "Msys" at the end. Then sed replaces all occurrences of QSP with those new functions. You are partially right about MSYS2. While Cygwin does everything inside emulated POSIX, MSYS2 only use emulation for "/usr". Additionally there are "/mingw32" and "/mingw64" paths where native software is placed ​https://sourceforge.net/p/msys2/wiki/MSYS2%20introduction/

Windows platform is a bit different and MSYS2 paths must be listed before Windows native paths. Implementing new functions with proper returns was an easy solution.

[RJVB (René Bertin)]

Then sed replaces all occurrences of QSP with those new functions

Meaning you have to patch *all* code you build against your patched Qt, with the risk of forgetting?

[mkae (Marko Käning)]

Replying to rjvbertin@…:

Meaning you have to patch *all* code you build against your patched Qt, with the risk of forgetting?

Would have been my question too. Every dependent package would now have to be patched to make use of these new MSYS2'ish Qt-QSP-functions... Doesn't sound practical.

[RJVB (René Bertin)]

Have another look at my QSP/XDG patch. I introduce class methods that take a mode switch, which is effectively hidden and hard-coded to "native" in the standard QSP usage. There's an extended QSP class in which those methods are exposed and the mode switch gets a default value that is determined at compile time. Sounds complicated (and it probably is a bit more so than strictly necessary), but in practice you just have to set 2 preprocessor tokens (macros) to build dependent code such that it uses the appropriate QSP mode. One macro basically replaces the QStandardPaths class with the QExtStandardPaths class, while the other macro determines the default value for the mode switch.

[mati865@…]

Replying to rjvbertin@…:

Then sed replaces all occurrences of QSP with those new functions

Meaning you have to patch *all* code you build against your patched Qt, with the risk of forgetting?

Yes but risk is reduced near to zero thanks to PKGBUILDs and makepkg (Arch Linux build tools ported to MSYS2).

[mati865@…]

Replying to rjvbertin@…:

Have another look at my QSP/XDG patch. I introduce class methods that take a mode switch, which is effectively hidden and hard-coded to "native" in the standard QSP usage. There's an extended QSP class in which those methods are exposed and the mode switch gets a default value that is determined at compile time. Sounds complicated (and it probably is a bit more so than strictly necessary), but in practice you just have to set 2 preprocessor tokens (macros) to build dependent code such that it uses the appropriate QSP mode. One macro basically replaces the QStandardPaths class with the QExtStandardPaths class, while the other macro determines the default value for the mode switch.

I think I've looked at wrong patch. ​https://github.com/RJVB/macstrop/blob/master/aqua/qt5-kde/files/fix-qstandardpaths4.patch is the correct one? It could be possible to extend this patch, however I don't full understand it yet. I'll add you do github discuss about patching Qt.

[RJVB (René Bertin)]

Yes, fix-qstandardpaths4.patch is the current patch.

[mkae (Marko Käning)]

Why did you split off kf5-frameworks-1.0.tcl from kf5-1.1.tcl ?

Strange, I never copied the latter into my MacPorts installation's portgroup folder... X-O

How come it worked anyways?

[RJVB (René Bertin)]

Replying to mk@…:

Why did you split off kf5-frameworks-1.0.tcl from kf5-1.1.tcl ?

I did some tinkering on a port (QupZilla) that has the option to use a specific KF5 framework, but that really cannot be called a "KF5 application". Specifically, it doesn't use cmake for configuring, but it also has no interest in the majority of other things that the KF5 PortGroup does. I had to introduce yet another control variable for the PortGroup, and use 3 of them from QupZilla's Portfile in order to be able to include the KF5 PortGroup without unwanted side-effects. All for just being able to declare a dependency on a single KF5 framework.

So I decided to create a special PortGroup that provides only the dependency features and that can be included without any side-effects.

I'm not aware of some guideline against justifiable PortGroups, is there?

Strange, I never copied the latter into my MacPorts installation's portgroup folder... X-O How come it worked anyways?

Simple: all KF5 ports are in the same tree as kf5-1.0.tcl so the file was always found. You only need to copy over PortGroup files if they are to be used by ports in a different ports tree. Like for instance if you want to use port:qt5-kde with mainstream Qt5 ports.

[mkae (Marko Käning)]

Replying to RJVB:

I did some tinkering on a port (QupZilla) that has the option to use a specific KF5 framework, but that really cannot be called a "KF5 application".

I see. Well, the reuse of KF5 libs by apps outside of KDE's ecosystem is actually encouraged, so I believe it's a good idea to have a ready made port group for it!

[mkae (Marko Käning)]

One important thing to consider soonish is, that we need to dump the folder kf5 after all, as all its KF5 port files finally need to be located in the kde folder.

[RJVB (René Bertin)]

Replying to mkae:

One important thing to consider soonish is, that we need to dump the folder kf5 after all, as all its KF5 port files finally need to be located in the kde folder.

Says who? I am really not in favour of that... The kde folder already has KDE3 and KDE4 ports.

But ultimately this is just a question of organisation. It shouldn't matter where I am hosting the port directories and where they'll end up after committing.

[mkae (Marko Käning)]

Replying to RJVB:

Says who? I am really not in favour of that... The kde folder already has KDE3 and KDE4 ports.

I remember that this was a response a while ago on MP's dev ML, but I can't find it anymore. :( The argument was that the KF5 belongs to KDE stuff just like KDE 3 and 4 did, thus those port files should live in there. But well, I could ask on the ML again.

But ultimately this is just a question of organisation. It shouldn't matter where I am hosting the port directories and where they'll end up after committing.

Yes, the only issue might be name duplicates, but they can be avoided with a "kf5-" prefix for the ports in question, or those "*5" postfixes.

[RJVB (René Bertin)]

Yes, I understand the argument, but there's also the question of maintainership. It's simply easier for Nicos and for me to do bulk changes (or only their search part) if we can limit the scope of such operations on a single directory. I don't see any arguments why using a single bulk directory would make anything easier for us.

Of course I don't know what arguments there might be against creating a new public category/directory but I cannot think of any that outweigh maintainer convenience.

[mkae (Marko Käning)]

OK, so the ML's conclusion looks like avoiding the separate kf5 directory.

We've got to start with qt5-kde anyways.

[RJVB (René Bertin)]

Huh, I'm not seeing any answers to your question?

[mkae (Marko Käning)]

vq stated: I think having top-level directories based on implementation details like language or framework are silly and pointless from a users' perspective, and we should not add anymore of them.

(That's what I am referring to.)

[RJVB (René Bertin)]

But the users are probably that last ones who're going to care ...

I think this question should never have been raised. It'd be a whole lot easier to move everything from kf5 into kde than vice versa.

[mkae (Marko Käning)]

I raised it because I wasn't sure whether there are other side effects. I've gotten similar response perhaps a year ago or so... Well, I don't want to ignore Lawrence's opinion, but I'd really like to get more feedback from the dev list!! So far too few people care. This might change once qt5-kde is up and running though.

[chrstphrchvz (Christopher Chavez)]

I was investigating if QupZilla can be updated to Falkon, which now uses CMake and requires ki18n. I may create a temporary port for ki18n, cf. ​https://github.com/macports/macports-ports/pull/2020

[mascguy (Christopher Nielsen)]

Add keyword portgroup, to pg-related tickets