qt: allow side by side installation of qt4-mac and qt5-mac

[mojca (Mojca Miklavec)]

As mentioned in #37331, please modify qt4-mac and qt5-mac to allow simultaneous installation of both.

[sicherha@…]

Cc Me!

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Just off the top of my head, there would seem to be four options:

1. Put both the Qts in their own private directories (as mentioned in comment:86:ticket:37331)
   • This is the default behavior (put everything in /usr/local/Qt-5.3.1 or /usr/local/Trolltech/Qt-4.8.6).
   • The binaries, cmake files, pkg-config files, and frameworks would be harder for scripts to find.
   • I believe qt4-mac used to do this prior to r72571.
2. Put one of the Qts in its own private directory for anyone who might need it and "bless'' the other as the "official" supported version by putting it in ${prefix}.
3. Try to figure out a way to get both installed in ${prefix}.
4. Keep things the way they are and force users to choose.

Options 1, 2, and 4 would be fairly easy to implement (4 would be the easiest).
Personally, I only need one version, so I have no strong feelings one way or the other.
Does anyone else have any thoughts?

[digitalriptide@…]

Cc Me!

[mojca (Mojca Miklavec)]

Anything but (4) is acceptable.

I'm sure that a lot of software doesn't work properly with Qt 5 yet. And some software doesn't work with Qt 4 any longer. We need to have both installed simultaneously to be able to do a soft transition and in order to be able to update the ports depending on Qt one-by-one.

The goal is to make everything work with Qt 5 at some point in future, but that transition will definitely take time.

(2) I would say that using a private prefix for Qt 5 is "good enough" for now unless the software would link against Qt 4 by accident (just because -I${prefix}/include contains Qt 4 for example).

(3) If everything would stay under ${prefix} we have problems at least with:

• ${prefix}/bin/*
• ${prefix}/include/Qt* (pointers to ${prefix}/Library/Frameworks/Qt*.framework/Headers)
• ${prefix}/include/Qt/*.h
• ${prefix}/Library/Frameworks/Qt*.framework/Versions/Current
• ${prefix}/lib/pkgconfig/*.pc
• ${prefix}/lib/*.(dylib|la|prl) (some of them links to ${prefix}/Library/Frameworks/Qt*.framework/Qt*)

I have a feeling that moving these files around kind of defeats the purpose of having these files there in the first place (and having both in a private folder would be just as fine). But if someone can find a solution, that would be great.

For the moment it would probably be acceptable to keep Qt 4 as the official version for a while, but put Qt to a private location (specified in the PortGroup and easily accessible via some variables), so that port maintainers could test their packages. What I don't know is whether this can be achieved without Qt 4 interfering.

So if (2) and (3) are not easily doable, we might have to go for (1).

Of course if anyone is brave enough to port everything to Qt 5 at once it would be ok to make a drastic move to Qt 5 only. But that would probably take too much effort.

[hsivank@…]

(1) +1
I use this method with macports and more than 5 different versions of Qt … it works great

(2) This is really not a good idea. If you have Qt4 in prefix, ALL ports based on Qt5 will fail to build (and vice & versa).
And i'm sure that all Qt port will not switch to the same version of Qt ...

(3) in fact : egual (2)
Think about headaches with Qt6 integration ...

[mkae (Marko Käning)]

Cc Me!

(3) +1

[mopihopi]

Cc Me!

[RJVB (René Bertin)]

I'm sure that a lot of software doesn't work properly with Qt 5 yet.

There's also software that's Qt5 only (like the current QtCreator versions). It's not like Qt5 is just out and still looking for adopters ...

Is there a reason why MacPorts wouldn't adopt the way Ubuntu (Debian) do things? They store Qt versions in their own prefix, but add qtchooser and qtN-default packages. The latter allows you to set up the symlinks needed to make a particular version the default, the latter links stuff through a chooser which has both a default and options to override that. For example:

390071 1 lrwxrwxrwx 1 root root 9 May  3 12:56 /usr/bin/qmake -> qtchooser*

I don't know how many Qt-related pkg-config scripts there are, but anything that goes through qmake will be able to find (and be found) automagically wherever Qt is installed. And nothing forbids ports from specifying a specific qmake command like qmake-qt4 (if qmake -qt=4 is infeasible).

[mojca (Mojca Miklavec)]

Replying to rjvbertin@…:

Is there a reason why MacPorts wouldn't adopt the way Ubuntu (Debian) do things? They store Qt versions in their own prefix

Manpower. Packaging of wxWidgets (which had a similar problem) has been changed last year. Qt would have to follow, but someone needs to write the code.

[ctreleaven (Craig Treleaven)]

Cc Me!

[IanWadham]

Just for information and not wishing to put on pressure, the KDE developers have started to ship KDE Frameworks 5, the new KDE libraries, which depend on Qt 5 (​https://techbase.kde.org/Schedules/Frameworks). In December they are planning to release a mixed batch of KDE apps (i.e. no libraries), which has been dubbed KDE 14.12 (yes, year.month). Some of these apps will have been ported to use Frameworks and Qt 5 and some (perhaps most) will still be using KDE 4 libraries and Qt 4. The releases of KDE apps will be named by year and month from then on.

No doubt the proportion of ported apps will grow over time, as it did with the KDE 3 to KDE 4 transition. Whatever happens, MacPorts will need to provide a mix of KDE 4, Qt 4, Frameworks 5 and Qt 5 for some time to come, with KDE apps building onto one dependency path or the other. The good news is that this change does not seem to be as "radical" as the KDE 3/4 change was, from the point of view of applications and API, but it will be quite a lot different in its build characteristics - hopefully a lot simpler once the changeover has been made.

[RJVB (René Bertin)]

Cc Me!

[RJVB (René Bertin)]

Replying to mojca@…:

Is there a reason why MacPorts wouldn't adopt the way Ubuntu (Debian) do things? They store Qt versions in their own prefix

Manpower. Packaging of wxWidgets (which had a similar problem) has been changed last year. Qt would have to follow, but someone needs to write the code.

Can it take that much manpower to add a --prefix /opt/wherever.qt5?

Wouldn't it actually be harder to get Qt5 to build for 10.6?

[mkae (Marko Käning)]

Replying to rjvbertin@…:

Wouldn't it actually be harder to get Qt5 to build for 10.6?

I see that you're able to build current KDE versions on 10.6 using gcc 4.8... Perhaps this offers the opportunity to also get qt5-mac build for 10.6?

[RJVB (René Bertin)]

That's actually on my TODO list ...

The other option would be somehow to limit Qt5 to 5.2.x on 10.6, supposing that version builds with less issues.

[RJVB (René Bertin)]

I just discovered another potential option.

When I tried to install Qt's own 5.3 binaries through their installer, this failed on the step that adapts the "rpath" information to the chosen install path (the shipped binaries have a very long _PADDING_ string in their path, for that purpose. However, they depend on libstdc++, so I could install to the intended location on my 10.9 VM, and then copy that tree to the 10.6 host. Works a charm. Evidently I could have picked a location in /opt/local.

I still haven't grasped what the QStandardPath issue is that Marko is working on, or if there are (other) modifications to the Qt source code to cater for use by MacPorts. But barring those, it seems MacPorts could host a binary package created on 10.7 or newer of an install to /opt/local/<whatever>/qt/5, with symlinks to the usual locations (and qtchooser to help have both 4 and 5 installed in parallel).

[mkae (Marko Käning)]

Replying to rjvbertin@…:

I still haven't grasped what the QStandardPath issue is that Marko is working on

Please check the "Standard paths for Qt5" section on KDEProblems/KDEMacPortsCI/Status for more info as well as ​David's reply on K-F-D.

[michaelld (Michael Dickens)]

The actual changes to make this happen should be pretty small: edit the qt portfiles and change the qtdir to something else that is unique to each Qt.

That said: I think the greatest challenge in this ticket's implementation is the sheer number of ports that depend on Qt (4 or 5) that will have to be worked over to make sure they are doing the right thing. Once the qtdir has been committed, there will be lots of ports that work correctly "out of the box". But, there will also be -lots- of ports that misbehave: some will not use qtdir / QMAKE correctly & have to be told specifically what to do; others will need to be patched to get them working since they hard-wire were QMAKE is located (or, something like that).

I think to do diligence on this, we need 2 or 3 of us devs willing to commit a week's time to fix issues once the primary commit is made, pretty much -just- fixing Qt issues. The next week, we can drop down to 2, then down to 1, and so forth. All in my opinion, of course; I'd love to be proven wrong!

I'm all for having side-by-side Qt installations; I just don't have the time right now to put in to make it so.

[RJVB (René Bertin)]

OK, I'll have to read up on the standard paths issue.... (when ​http://qt-project.org/doc/qt-5/qstandardpaths.html is back up. ..) But as to xdg ... what's to keep us from having configuration files under ${prefix}/etc/xdg(/qtchooser/*.conf) ? There seem to be a few xdg/freedesktop related ports already, after all ...

As to the parallel installation issue, I agree with Michael. Linux packagers have found a certain number of solutions, among which qtchooser (​http://manpages.ubuntu.com/manpages/trusty/en/man1/qtchooser.1.html).

I cannot evaluate how many ports there are now that depend on Qt5, but I would guess that there are (many) more ports that require Qt4 than Qt5. (Edit: I count 6 Portfiles that contain the string qt5, apart from qt5-mac and qt5-creator ... and that includes pyqt5...). So the most safe approach would probably be to leave Qt4 where it is as far as that's possible OR by default link tools like qmake to the Qt4 version.

There might also be the option to use versioned frameworks, but I have never understood how (and if...) that actually works without specifying all paths by hand...

[mkae (Marko Käning)]

Replying to rjvbertin@…:

what's to keep us from having configuration files under ${prefix}/etc/xdg(/qtchooser/*.conf) ? There seem to be a few xdg/freedesktop related ports already, after all ...

Nothing really. It's perhaps just more platform-compliant to have all those configs in ~/Library/Application Support/ like KDE currently saves stuff in ~/Library/Application Support/KDE/ .

For the OSX/CI system I am currently trying to avoid this still and stick to the XDG stuff, because that's what the CI scripts give me at the moment - although this is NOT recommended by David Faure in the mentioned K-F-D thread.

W.r.t to standard paths check out ​the KDE frameworks migration epics page.

[quantum7@…]

Cc Me!

[jeabraham]

Cc Me!

[supertwang]

Cc Me!

[springermac (Jonathan Springer)]

Cc Me!

[hmeine (Hans Meine)]

Cc Me!

[Greisby (Greisberger Christophe)]

Cc Me!

[davidwalton3142@…]

Cc Me!

[joergahrens (Jörg Ahrens)]

Cc Me!

[BSeppke (Benjamin Seppke)]

Cc Me!

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

As discussed previously, qt5-mac now installs in private directory (see r137229).

[RJVB (René Bertin)]

Replying to mcalhoun@…:

As discussed previously

That doesn't refer to discussion other than the one above, to install it *elsewhere* than I have been proposing, I hope?! Rather surprising (or not...) that this gets committed without even taking up the slighest detail with me...

Just a few of the remarks I'd have made:

• Qt 5.4.2 is out ...
• no need to call the installation directory qt5-mac . The whole distinction qtN-mac vs qtN-somethingElse has become moot now that Qt moved to the QPA plugin approach and does not officially support non-native mode on OS X
• depending on PulseAudio is *not* required for anything (that I know of), not anymore than building the internal phonon stuff.
• why add -qml-debug
• probably more but I'm on a clock right now

[dliessi (Davide Liessi)]

Cc Me!

[hmeine (Hans Meine)]

Replying to mcalhoun@…:

As discussed previously, qt5-mac now installs in private directory (see r137229).

That sounds great. Even if I understand that the port can still be improved, this sounds like it will finally help many many MacPorts users who are willing to install Qt5, but still absolutely need Qt4 for various reasons.

However, the port is still marked to conflict with qt4-mac, am I wrong?

[michaelld (Michael Dickens)]

Now that qt5-mac installs privately, I think it's time for us to move qt4-mac to do the same. This'll be somewhat of a headache, so I don't want to do it until I'm available to handle the fallout. RIght now I'm on vacation, so let's aim for doing this June 29 when I'm formally back at work. We'll need to rev-bump a bunch of ports that depend on the qt4 and/or qmake PortGroup. I'm quite sure it'll take some time to work out the kinks, but this way it's done. Yes, it's about time.

[RJVB (René Bertin)]

I have already identified a (surprisingly!) few ports that are ill-behaved and will break (corrected versions are available), and I'll be more than willing to help with other fallout. Provided of course that the transition is based on my work, and that at least some of my other changes are taken into consideration too. Off the top of my hat, there are the exception-less build (reducing overhead and bringing qt4 on the same page as qt5 in that aspect), the patches for improved font style support, and those improving the KDE4 experience. And there is the return of the sqlite3 plugin back into the main port of course, so that one doesn't have to install an auxilliary port before being able to test all of the few end-user applications qt4-mac installs (Bradley's suggestion, and I fully agree with it).

I have a fully working qt4-mac-devel port that can be used for testing right now, with the changes to the PortGroup done in such a way that they do not interfere with the current installation. In other words, it detects if qt4-mac was installed in what I call "exclusive" mode and sets up the environment appropriately. If so required, I can modify it such that this is the default more also when building qt4-mac itself, for now. My qt4-mac-devel port also has a subport that allows anyone to install it WITHOUT having to rebuild all qt4 dependents (and that one has been tested thoroughly because I myself didn't want to have to go through such a full rebuild while testing).

Isn't there something like a buildbot that could be set to run a rebuild of all the dependent ports without forcing them upon the world, in order to take stock of the fallout? Bradley (pixilla) has been asking me for such an action. As I said elsewhere, I'm not sure if it's really up to the qt4-mac maintainer(s) (and/or Bradley and me) to do this kind of thing, but I also see no issue with it provided we can use an existing infrastructure to (re)build all dependents without first having to identify and install them more or less by hand (and on a VM that I think severely under-resourced for the purpose).

[RJVB (René Bertin)]

Oops: the only (ok, other) unknown here is the time period. Michael returns on or after June 29th, I myself have little knowledge of my availability over summer. I probably should be available up to and part of the 1st week of July though.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Replying to rjvbertin@…:

Just a few of the remarks I'd have made:

• Qt 5.4.2 is out ...
• no need to call the installation directory qt5-mac . The whole distinction qtN-mac vs qtN-somethingElse has become moot now that Qt moved to the QPA plugin approach and does not officially support non-native mode on OS X
• depending on PulseAudio is *not* required for anything (that I know of), not anymore than building the internal phonon stuff.
• why add -qml-debug
• probably more but I'm on a clock right now

• qt5-mac has now been upgrades to 5.4.2 (r137535)
• Calling the directory qt5-mac might have been unnecessary. It is just aesthetics?
• The configure script uses PulseAudio if it is found (hence the dependency). It is possible to force the script not to look for it, but I tried to stay close to default behavior.
• Similarly, qml-debug is used because it is the default behavior.

[RJVB (René Bertin)]

Replying to mcalhoun@…:

• Calling the directory qt5-mac might have been unnecessary. It is just aesthetics?

More a matter of principle and consistency. The port itself could have been called qt5, and probably should have.

• The configure script uses PulseAudio if it is found (hence the dependency). It is possible to force the script not to look for it, but I tried to stay close to default behavior.

Well, the default behaviour is not to install PA and then use it if it's found... The logic behind the default behaviour is undoubtedly not to oblige people to enable the PA support on systems where PA is used and a usual alternative to ALSA (i.e. Linux). The default on OS X would be that PA isn't found, and thus support not built in. It's only MacPorts that imposes to make a choice, but I continue to think the logical choice here would be to hard-code that effective default. For multiple reasons in addition to the fact that using PulseAudio doesn't make sense for applications and middleware that already know how to use CoreAudio :

• PA causes a non-negligible number of dependencies to be pulled in, including part of gnome
• I don't think that the PA port does anything to ensure the PA daemon is started on login
• I'm not convinced at all that Qt applications will use PA even when the daemon is running. In fact, the only use of PA from Qt apps I've seen to date are through Phonon and the phonon-gstreamer backend.

• Similarly, qml-debug is used because it is the default behavior.

What's the point in hard-coding default options if they don't have implications w.r.t. dependencies? It only increases the workload at each upgrade, having to check which options have changed defaults. I don't know how generally useful this option is, however, nor what overhead it carries.

[quantum7@…]

Cc Me!

[digitalriptide@…]

Cc Me!

[digitalriptide@…]

Cc Me!

[michaelld (Michael Dickens)]

Slowly making progress on updating qt4-mac dependent ports to work when qt4-mac is installed other than ${prefix}. Most require just a rev-bump, but some are stubborn, mostly because phonon is assumed to be part of Qt4 but in MP it is not & the ports are not doing the right thing (by assuming, instead of checking via pkgconfig or whatever). When a rev-bump is not enough, I'll directly push updates to the port (if nomaintainer) or create a ticket (if owned at all, whether openmaintainer or not). This process will take a few days, so I expect to do "the big commit" sometime next week (July 6-10). Your patience is appreciated.

[hmeine (Hans Meine)]

And we do appreciate your intermediate progress update!

[RJVB (René Bertin)]

Replying to michaelld@…:

mostly because phonon is assumed to be part of Qt4 but in MP it is not & the ports are not doing the right thing (by assuming, instead of checking via pkgconfig or whatever).

Hmmm, didn't run into that particular issue; KDE4 ports seem to be fine in that aspect.

Are you following my layout, or just dumping everything into a private prefix as mcalhoun did for qt5-mac? Moving the Qt plugins directory from where it was might explain the phonon issue, and could well be what made me look once more and decide there was no point in changing install locations that were already compatible with concurrent Qt 4,5 installation.

[michaelld (Michael Dickens)]

Yes, moving Qt4 (back) into libexec, just like Qt5 is now. It's easier to debug issues when we know where the install is supposed to be located. Ditto for dependencies, and so forth. MacPorts is a control freak ;)

BTW> Do we want the Qt4 install prefix to match that for Qt5, and be ${prefix}/libexec/qt4-mac? I was just going to use: ${prefix}/libexec/qt4 for simplicity, since I cannot foresee a time when qt4-x11 is a viable alternative. Nor qt5-x11 either, for that matter. Hence, just using "qt4" as my preference. Thoughts?

[RJVB (René Bertin)]

I think I made my thoughts on this abundantly clear. There's nothing easier about moving everything in a single prefix as opposed to moving only the required bits to ${prefix}/libexec/qtN, and certainly less to debug because we already know where everything (else) is supposed to be located. And those are exactly the bits about which dependent code (as opposed to dependent build systems) may make assumptions.

qt5-x11 is a perfectly viable runtime alternative, but I'm not going to be coming back to that.

As to control freaking, I might simply break out my qtN-kde subports into full ports with a single maintainer (but will most likely keep them to myself).

[michaelld (Michael Dickens)]

Given the transition for many folks to 10.11 & the need for many folks to build/rebuild qt4-mac because of the issues from ticket #48129, I'm also thinking of doing the install location transition here at the same checkin time. If things are going to be (potentially) messy, let's just get it over with (IMHO)! I'm going to give folks through the weekend for testing / evaluation of the patch from the other ticket & then commit on Monday if there are no issues discovered. Constructive thoughts or comments (e.g.: beyond "it's about time"; or "my thoughts are abundantly clear"; etc).

[RJVB (René Bertin)]

Do you even have to ask, I'd hope my thoughts on this have been abundantly clear for months now :P

I'll bundle and test the 10.11 patch (on 10.9) with my own qt4-mac-devel port and upload a tarball of it here, as a constructive gesture (realising full well only thoughts and comments were sollicited :P)

[RJVB (René Bertin)]

Replying to rjvbertin@…:

I'll bundle and test the 10.11 patch (on 10.9) with my own qt4-mac-devel port and upload a tarball of it here

Actually, no need to apply a patch to the code on OS X < 10.11 ...

[RJVB (René Bertin)]

Here's my polished-up port:qt4-mac-devel, this time bundled in a single Portfile. It uses minimal modification of the current, exclusive, install layout, and also introduces a number of other changes among which patches that improve behaviour of KDE4 applications (without degrading that of pure Qt apps). It also allows for a leaner build, esp. with LTO.

[michaelld (Michael Dickens)]

Replying to rjvbertin@…:

Replying to rjvbertin@…:

I'll bundle and test the 10.11 patch (on 10.9) with my own qt4-mac-devel port and upload a tarball of it here

Actually, no need to apply a patch to the code on OS X < 10.11 ...

True: no need. But, the patch won't hurt & it will then be compatible across all OS/version installs. So, I'm going to include it for everyone.

[RJVB (René Bertin)]

This has nothing to do with cross-OS-version compatibility ... A quick look at the code modifications suggest that the patched version could well be less efficient, and if that's the case I don't see the point in imposing it needlessly.

[michaelld (Michael Dickens)]

Replying to rjvbertin@…:

I'll bundle and test the 10.11 patch (on 10.9) with my own qt4-mac-devel port and upload a tarball of it here, as a constructive gesture (realising full well only thoughts and comments were sollicited :P)

OMG that's more than what I was asking for. You knew that, too ;P}

What are the primary/major differences between your setup and the current on in MP?

-> Maybe: Send me a private email naming each new patch file & new variant and briefly what it addresses / does? I'm really not sure how to approach all of your changes!

-> You know that I'm not going to just use what you provide out of the box. That said, you've put a lot of time into these changes and I'm quite sure that some of them fix bugs or add usable features -- so, I'd like to determine which do what. But, this is a big task & my time is limited.

-> NOTE: I don't want a formal diff (e.g., "svn diff") between the current Portfile and yours; just too many changes to be able to parse what is what. Nor do I care about the qt_dir install prefix, or parallel install, or transitional / exclusive or anything along those lines. We're going to move qt4-mac into ${prefix}/libexec/qt4-; period.

-> For example, I notice quite a number of patches added to yours, and some extra variants. Do these fix bugs in open MP (or other) ticket and/or add usable functionality? Do they allow for better building across more compilers / OSs (e.g., KDE, X11, Aqua)?

[michaelld (Michael Dickens)]

Replying to rjvbertin@…:

This has nothing to do with cross-OS-version compatibility ... A quick look at the code modifications suggest that the patched version could well be less efficient, and if that's the case I don't see the point in imposing it needlessly.

Well, this is the way the Qt folks do it in Qt5; I just back-ported it to Qt4. I don't care if it is more or less efficient. If it's good enough for Qt5 then it's good enough for Qt4 (IMHO).

[RJVB (René Bertin)]

Replying to michaelld@…:

Well, this is the way the Qt folks do it in Qt5; I just back-ported it to Qt4. I don't care if it is more or less efficient. If it's good enough for Qt5 then it's good enough for Qt4 (IMHO).

You may be right, but Qt5 has lots of other changes under the hood which may make this particular code path much less of a bottleneck that it *maybe* is in Qt4

[michaelld (Michael Dickens)]

Replying to rjvbertin@…:

Replying to michaelld@…:

Well, this is the way the Qt folks do it in Qt5; I just back-ported it to Qt4. I don't care if it is more or less efficient. If it's good enough for Qt5 then it's good enough for Qt4 (IMHO).

You may be right, but Qt5 has lots of other changes under the hood which may make this particular code path much less of a bottleneck that it *maybe* is in Qt4

Also true. Bottom line is that this patch allows Qt4 to build on 10.11 without resorting to using the 10.10 SDK; it seems to work correctly too. So, no matter the performance, this is a win at least for 10.11 users (who will be numerous shortly, since Apple just gives away the OS). I also doubt that the performance is very different between the original and this version, as the primary change is in "QCoreGraphicsPaintEngine::clearColorSpace", which, really, isn't likely to be used very often (do a grep though the qt4 source & you'll find a few references to this; grep those & you find almost no references). This method was removed in Qt5 sometime; apparently it really isn't used much if at all. Thus, I don't worry about performance, either.

[RJVB (René Bertin)]

Replying to michaelld@…:

Also true. Bottom line is that this patch allows Qt4 to build on 10.11 without resorting to using the 10.10 SDK; it seems to work correctly too. So, no matter the performance, this is a win at least for 10.11 users

Of course, but wrapping it in a conditional in the Portfile also isn't a huge deal. And something the Qt devs couldn't do with their build system ;)

[RJVB (René Bertin)]

Replying to michaelld@…:

What are the primary/major differences between your setup and the current on in MP?

The one currently in MP being the "exclusive" version? The main difference with that install layout is the fact that libraries and executables are moved into ${prefix}/libexec/qt4. Everything that was already under a qt4 directory (share/qt4, doc/qt4) remained there (with maybe a few things moving around, but as little as possible).

-> Maybe: Send me a private email naming each new patch file & new variant and briefly what it addresses / does? I'm really not sure how to approach all of your changes!

Erm, I think I have already done that a long time ago, about the time when I considered this port "done".

-> You know that I'm not going to just use what you provide out of the box.

Why do you think I provide this as a "-devel" port? Yes, over time lots of changes have accumulated (and they should all be documented at least in the naming of patch files). I would in fact invite you to build this baby and see what comes out of that, and my offer stands to help out with any issues that arise from my work. port:qt4-mac is openmaintainer, and if I'd had commit powers I'd have pushed qt4-mac-devel a long time ago already.

-> NOTE: I don't want a formal diff (e.g., "svn diff") between the current Portfile and yours; just too many changes to be able to parse what is what.

I just included one because I was certain someone was going to insist on having it (and then telling me it's unreadable) :)

Nor do I care about the qt_dir install prefix, or parallel install, or transitional / exclusive or anything along those lines.

Transitional was always just that, and easy to strip. I'm not even sure to what extent I myself still have ports that are built against the exclusive version.

We're going to move qt4-mac into ${prefix}/libexec/qt4-; period.

If you mean that you plan to move anything into libexec/qt4-mac then I fear the only thing left that we might agree on is the possibility of a qt4-kde port analogous to port:qt5-kde I submitted the other day, which I'd maintain. I'm not interested in having to find out what issues I ran into with that all-in approach, but I'm sure it had to do with my KDE4 ports.

-> For example, I notice quite a number of patches added to yours, and some extra variants. Do these fix bugs in open MP (or other) ticket and/or add usable functionality? Do they allow for better building across more compilers / OSs (e.g., KDE, X11, Aqua)?

I have not sought to address open MP tickets unless pointed to them specifically. But yes, every bit I added was to improve something:

• the sqlite3 plugin was rolled back into the main port as it's required by some of the applications the main port installs (suggestion from Bradley Giesbrecht)
• the default is to build without internal support for C++ exceptions except for the few modules which actually use them. That provides a leaner build with less overhead
• an LTO variant provides a build with link-time optimisation. This again gives a leaner build with snappier performance. (This and the previous item were achieved with guidance from Thiago Macieira)
• installation of the >200Mb of html docs is moved to a variant
• a KDE variant addresses a number of issues that KDE4 apps/users (can) run into (much) more frequently (than pure Qt apps).
• qprocess-nozombies.patch prevents zombie processes that could otherwise be left by QProcess
• qt4-prevent-mousevent-for-deleted-object.diff : does what it says; this could lead to crashes in KDE4 apps due to pending events delivered to deleted objects
• *exceptions* and *lto* patches have to do with the no-exceptions and LTO builds
• patch-tools-build_qtconfig.diff : builds the qtconfig utility
• patch-improve-fontweight-support2.diff, patch-fontpanel.diff : implement somewhat improved support for font weights other than regular/bold/italic/bold-italic
• qt4-deactivate-menurole-heuristics.patch (+KDE) : disables part of the text-heuristics based guessing which menu QAction should become the About or Preferences menuitem, That shortcut is rather flaky which has led to annoying misses in KDE4 apps; the patch is included in the KDE variant only.
• debug-negative-qtimerint.patch (+KDE): gives more useful feedback when a QTimer is started with a negative interval, something I've seen happen in KDE4 apps

Other patches in the KDE variant prevent null-pointer dereference in QAction::isEnabled() (undoubtedly again due to pending event handling), make QFileSystemWatcher less verbose and add a means to keep track of the number of open file descriptors when using that class (relevant for KDevelop).

If I missed anything just ask.

I see that I left in a reference to qt4-mac-devel-x11 . That's a remnant of attempts to experiment with QPA support which clearly isn't finalised in Qt4. So no Qt4-X11 support I'm afraid. I also haven't had any reason to experiment with other compilers; I have done so with my Qt5 ports however, so maybe some of that knowledge can be applied here. If there's a reason...

[RJVB (René Bertin)]

Annoyingly I'm headed for a busy weekend in which I won't have much time for dev issues ...

[mojca (Mojca Miklavec)]

Just a bit of teasing: After years of no activity on the Qt5 front and a year and a half of having conflicting Qt4/5 ports (basically: useless Qt5 port as long as one needed Qt4 somewhere) it is nice to see that a small and "stupid" incompatibility in 10.11 can finally start make the things moving ... ;)

Truth to be told, it would be extremely helpful if we had more time to prepare for this event, but from what I understand all Qt dependents (both Qt4 and Qt5) are currently broken on 10.11, so fixing anything is of a relatively high priority. We probably cannot afford to spend months testing all the Qt dependents, so the scenario will probably be as follows(?):

• warn all users (and developers) to avoid upgrades for the next X days if they are not willing to cope with consequences
• do the radical changes in qt4-mac and qt5-mac (maybe we could even call those ports qt4 and qt5 and leave the old ports there during the transition period; then remove them)
• ask everyone to fix as many dependent ports as quick as possible

Or do you have better ideas about how to go on with all the shifting of Qt4 files?

It takes days (certainly more than 24 hours) on my old mac to compile anything Qt-related, so testing multiple different installation options is an extremely painful process for me. But given how much effort René already put into trying to get things fixed also for KDE, it would be extremely helpful to take a careful look into his patches. If keeping some kind of consistent structure is important for KDE, I would be in favour of trying to obey at least some general guidelines and similarities with GNU/Linux to simplify compatibility. Having to have another copy of qt4-kde just for the sake of "more consistent" directories would be highly suboptimal and lots of extra work for everyone involved in Qt-related activities. And lots of extra disk space and precious processor time required.

[RJVB (René Bertin)]

Thank you Mojca (I guess :)) I've been typing this reaction over the course of breakfast (and "helped" by a couple of cats in need of attention :)) so it may not be as well structured as it could.

First off, existing users can be warned not to upgrade to 10.11, but I guess you also have newcomers in mind (who'd have to be told to wait until compatibility issues are fixed).

port:qt5-mac has already been made concurrent, so it would only require a patch if the commit Michael has mentioned has not made it to any released version. It would certainly surprise me if it hadn't made it into 5.5.0 (released not more than a few weeks ago), and that version is already available in the port:qt5-kde I submitted. Now to debunk a few misconceptions: the *-mac and proposed *-kde versions are not co-installable, so there is no *extra* disk space or processor time required. One advantage in my packaging: I don't build all of Qt5 in the main port, but moved the biggest hog (QtWebEngine, a new component in 5.4 and still little used) to a subport. And a hog it is.

There aren't many qt5 dependent ports ATM, but those I tried build and work just fine against port:qt5-kde . Sadly the *-mac and *-kde versions are not binary drop-in replacements but require a rebuild of all dependent packages.

• I could however add a variant like the "transitional" variant in my qt4-mac proposals, which puts up symlinks so that existing dependents continue to find the stuff they need under libexec/qt5-mac. I won't have much time for that this weekend, and would need someone to help testing in any case.

About the install layout/directory structure: I've seen it argued that just putting everything into a single directory is how Digia do it with their own installers, on OS X. True, but the use case they envision is developers who build standalone app bundles that can be submitted to the App Store. That's not the reason for which MacPorts contains Qt, I think. There are Qt ports so there can be any number of other ports to depend on the appropriate Qt version. Most of those dependents probably come from other Unix desktops where Qt does not install in a single directory.

In short, even if it's the easiest way to achieve Qt concurrency on the short term, it is IMHO *NOT* the appropriate way for something like MacPorts. And in fact (and again, IMHO) it is just as easy to change only the locations that require changing, esp. since just about all configurable install locations are already specified through dedicated variables in the PortGroup. Not everyone may know this but yes, Qt's configure script has options for a whole slew of installation locations, and I think we use about all of them. Final note: with the install layout I use I get:

> qmake-qt5 -query
INCLUDEPATH:/opt/local/include
INCPATH:/opt/local/include
--> QMAKE_LIBDIR:/opt/local/lib
QT_SYSROOT:
--> QT_INSTALL_PREFIX:/opt/local
QT_INSTALL_ARCHDATA:/opt/local/libexec/qt5
QT_INSTALL_DATA:/opt/local/share/qt5
QT_INSTALL_DOCS:/opt/local/share/doc/qt5
QT_INSTALL_HEADERS:/opt/local/include/qt5
QT_INSTALL_LIBS:/opt/local/libexec/qt5/Library/Frameworks
QT_INSTALL_LIBEXECS:/opt/local/libexec/qt5/libexec
QT_INSTALL_BINS:/opt/local/libexec/qt5/bin
QT_INSTALL_TESTS:/opt/local/share/qt5/tests
QT_INSTALL_PLUGINS:/opt/local/share/qt5/plugins
QT_INSTALL_IMPORTS:/opt/local/share/qt5/imports
QT_INSTALL_QML:/opt/local/share/qt5/qml
QT_INSTALL_TRANSLATIONS:/opt/local/share/qt5/translations
QT_INSTALL_CONFIGURATION:/opt/local/etc/qt5
QT_INSTALL_EXAMPLES:/Applications/MacPorts/Qt5/examples
QT_INSTALL_DEMOS:/Applications/MacPorts/Qt5/examples
--> QT_HOST_PREFIX:/opt/local
QT_HOST_DATA:/opt/local/share/qt5
QT_HOST_BINS:/opt/local/libexec/qt5/bin
QT_HOST_LIBS:/opt/local/libexec/qt5/Library/Frameworks
QMAKE_SPEC:macx-clang
QMAKE_XSPEC:macx-clang
QMAKE_VERSION:3.0
QT_VERSION:5.5.0

I highlighted some that will point to ${prefix}/libexec/${qt_dir} when using an approach as used by the current port:qt5-mac . It seems evident that the values above are to be preferred in a context like MacPorts'. Qt4 qmake is less expressive, and it looks like I'll want to double-check QT_INSTALL_PREFIX:

> qmake-qt4 -query
QT_INSTALL_PREFIX:/opt/local/libexec/qt4
QT_INSTALL_DATA:/opt/local/share/qt4
QT_INSTALL_DOCS:/opt/local/share/doc/qt4
QT_INSTALL_HEADERS:/opt/local/include/qt4
QT_INSTALL_LIBS:/opt/local/libexec/qt4/lib
QT_INSTALL_FRAMEWORKS:/opt/local/libexec/qt4/Library/Frameworks
QT_INSTALL_BINS:/opt/local/libexec/qt4/bin
QT_INSTALL_PLUGINS:/opt/local/share/qt4/plugins
QT_INSTALL_IMPORTS:/opt/local/share/qt4/imports
QT_INSTALL_TRANSLATIONS:/opt/local/share/qt4/translations
QT_INSTALL_CONFIGURATION:/opt/local/etc/qt4
QT_INSTALL_EXAMPLES:/Applications/MacPorts/Qt4/examples
QT_INSTALL_DEMOS:/Applications/MacPorts/Qt4/demos
QMAKE_MKSPECS:/opt/local/share/qt4/mkspecs
QMAKE_VERSION:2.01a
QT_VERSION:4.8.7

Side-note: I've submitted a few scripts the other day that I find very useful when doing incremental (re)builds of ports.

As to Qt4 : here the issue is much simpler really, partly also because building Qt4 is incomparable to Qt5. On my 4yo 2.7Ghz i7 (2*2 cores) with 12Gb and a HDD, I'd say it takes about 2h (using a stiff nice value) when I'm doing other light stuff. More with LTO of course, but still fully acceptable. And of course, clever use of -n, -o and -k helps speed up rebuilds tremendously.

Once again, the Qt4 port I attached here is set up so it can serve as a drop-in replacement for the current, "exclusive" port:qt4-mac. Full drop-in compatibility requires installing the transitional subport which caters to existing, non-rebuilt dependents. It's a little bit tricky to install when automatic rev-upgrade is allowed to rebuild ports, but once in place, anyone can do all the testing s/he requires. Rebuilding dependents one by one and of course including modifying the Portfile and rebuilding the port.
NB: rebuilding a dependent even *with* the transitional subport in place removes the need for the subport for the rebuilt dependent (because the linker stores resolved paths in the rpath, not the symlinks). And of course if you prefer to bite the bullet: there's no obligation to use the transitional subport.

• I cannot imagine an "easier" way to approach the problem at hand (surely there's a reason it's been put off for so long). If there weren't the fact that the offered solution has been tested since January (using KDE4 and pure Qt dependents) even I might think I'd prefer to just do it myself (when in Michael's shoes). But with that testing, and given the minimal changes to the tried-and-true install layout I'd realise soon enough that I'd be foolish to want to reinvent the wheel without as much as trying the proposed solution (esp. since I surely left the port openmaintainer because I never intended to be the sole Qt saviour in the MacPorts universe ;)).

We all realise there are lots of Qt4 dependents. The KDE4 ports have already been handled through some minor changes to the KDE4 PortGroup (but even I cannot claim I've test-built all of them). But that still leaves plenty of candidates that might not be well-educated (declare their Qt4 dependency throught the Qt4 PortGroup, and use its variables instead of hard-coded paths). I have tackled those I came across. I am 90% certain that I filed tickets for those, and maybe I'll get around to checking some more this weekend (all I need to do is deactivate the transitional subport to see which ports require rebuilding).

• But there must be more. I don't think they're going to be a challenge, just monks' work to get right.
• In fact, I'd see this as an argument just to commit qt4-mac-devel. It won't be installed automatically on any user machines, but devs like us can chose it deliberately over qt4-mac . Thanks to rev-upgrade a rebuild will be triggered of all installed dependents if the transitional variant is not installed. With a few devs doing this the problem ports will be identified soon enough.
  

Whatever happens after that period of getting kinks out of cables is something that can be decided without time pressure (and who knows, maybe you'll get me to admit I was wrong here or there O:^) ).

[RJVB (René Bertin)]

Ticket I mentioned with the convenience scripts: ​https://trac.macports.org/ticket/49009

[mojca (Mojca Miklavec)]

Replying to rjvbertin@…:

First off, existing users can be warned not to upgrade to 10.11, but I guess you also have newcomers in mind (who'd have to be told to wait until compatibility issues are fixed).

When I talked about upgrading, I meant upgrading packages on MacPorts. I'm almost sure that once we move Qt4 away from the default location there will be just a zillion of ports broken (disclaimer: that might not be the case with that "transitional" solution perhaps). Some might be fixed automagically be revbumping them, but I would expect that a lot of other ports would have to be fixed to continue working with the new Qt4 structure. A single Qt4/5 maintainer simply cannot fix all the ports, no matter what. (When I change the packaging of wxWidgets, it probably took me 2 weeks almost full time to sort everything out. And even then I managed to hurt some maintainers who thought that I simply made too heavy changes to their ports when I tried to make them compatible with 3.0.) So I expect a large number of ports to be broken after Qt4 gets updated. But developers won't fix their ports until a new version of qt4-mac comes out, ideally one that comes as a binary package. (Not upgrading to 10.11 is not the answer for everyone. And people who do upgrade right away need to be aware of potential problems with a bunch of software. That has always been the case with early adopters.)

Btw: if we are moving the complete structure anyway, what about renaming the new port simply to qt4? Now that x11 is hardly support (and probably not needed), there is no way to keep calling that port "mac". That way we could keep the old qt4-mac in place for a while (and ports that won't be upgraded yet might conditionally keep working), while urging everyone to switch to the new qt4 asap and update the ports to use qt4 instead of qt4-mac?

port:qt5-mac has already been made concurrent, so it would only require a patch if the commit Michael has mentioned has not made it to any released version. It would certainly surprise me if it hadn't made it into 5.5.0 (released not more than a few weeks ago), and that version is already available in the port:qt5-kde I submitted. Now to debunk a few misconceptions: the *-mac and proposed *-kde versions are not co-installable, so there is no *extra* disk space or processor time required.

Which is even worse. Because then I have no option to link, say, gnuplot against qt5-kde if I happen to like the kde structure more. Becasue that will make gnuplot conflict with just about every other Qt-based package in MacPorts.

There aren't many qt5 dependent ports ATM

I maintain a bunch of Qt-based ports that all work with Qt5, but I cannot afford to keep and test compatibility against Qt5 or get rid of Qt4 completely as long as qt5-mac is conflicting with qt4-mac.

• I could however add a variant like the "transitional" variant in my qt4-mac proposals, which puts up symlinks so that existing dependents continue to find the stuff they need under libexec/qt5-mac. I won't have much time for that this weekend, and would need someone to help testing in any case.

May I perhaps suggest you to put your files to a place like GitHub or any other public service, so that it becomes slightly easier to track your changes for those who might be interested? Uploading zips to trac starts getting cumbersome at some point, particularly with the amount of changes that you need to make.

About the install layout/directory structure: I've seen it argued that just putting everything into a single directory is how Digia do it with their own installers, on OS X. True, but the use case they envision is developers who build standalone app bundles that can be submitted to the App Store. That's not the reason for which MacPorts contains Qt, I think. There are Qt ports so there can be any number of other ports to depend on the appropriate Qt version. Most of those dependents probably come from other Unix desktops where Qt does not install in a single directory.

Quick question: do I understand it properly that some of your ports (mostly those based on KDE) do not even work if one installs the complete Qt into a single directory like libexec? If that is the case, we might have a strong argument to try to keep the directories spread out at their usual location. Or do both configurations (either a single directory or different directories) work with most ports?

Once again, the Qt4 port I attached here is set up so it can serve as a drop-in replacement for the current, "exclusive" port:qt4-mac. Full drop-in compatibility requires installing the transitional subport which caters to existing, non-rebuilt dependents.

So if I understand properly, installing both ports (your new qt[45]-mac) as well as the transitional subport will make sure that all the ports that haven't been fixed and/or rebuilt yet will happily keep working? And the transitional subport will only be a problem when building other ports against qt5? If that is so, the transition might be a lot less painful than what I imagined.

• But there must be more. I don't think they're going to be a challenge, just monks' work to get right.
• In fact, I'd see this as an argument just to commit qt4-mac-devel. It won't be installed automatically on any user machines, but devs like us can chose it deliberately over qt4-mac . Thanks to rev-upgrade a rebuild will be triggered of all installed dependents if the transitional variant is not installed. With a few devs doing this the problem ports will be identified soon enough.
  

Potentially the qt4-mac-devel could be called qt4?

[RJVB (René Bertin)]

Replying to mojca@…:

Replying to rjvbertin@…:

First off, existing users can be warned not to upgrade to 10.11, but I guess you also have newcomers in mind (who'd have to be told to wait until compatibility issues are fixed).

I would expect that a lot of other ports would have to be fixed to continue working with the new Qt4 structure.

The vast majority of the ones I've installed simply rebuild because 1) they use the Qt4 portgroup and 2) my layout changes are so minimal.

Btw: if we are moving the complete structure anyway, what about renaming the new port simply to qt4? Now that x11 is hardly support (and probably not needed), there is no way to keep calling that port "mac". That way we could keep the old qt4-mac in place for a while (and ports that won't be upgraded yet might conditionally keep working), while urging everyone to switch to the new qt4 asap and update the ports to use qt4 instead of qt4-mac?

Something I proposed for the Qt5 port and that got no traction at all. Besides, if ever we do end up with qt*-kde ports there'd actually be a reason for the -mac suffix ...

Which is even worse. Because then I have no option to link, say, gnuplot against qt5-kde if I happen to like the kde structure more. Becasue that will make gnuplot conflict with just about every other Qt-based package in MacPorts.

No. I've taken care that installing qt5-kde does not get you into conflict with binary packages built against qt5-mac. The other way around is less sure, but can be protected the same way. If you prefer the kde structure, you'll just end up using that structure for all Qt dependents, and I think it'll be possible to get binary packages for at least some of those (the ones conceived to be used with the kde structure in any case).

I maintain a bunch of Qt-based ports that all work with Qt5, but I cannot afford to keep and test compatibility against Qt5 or get rid of Qt4 completely as long as qt5-mac is conflicting with qt4-mac.

Which still boils down to the same thing, at least if I understand correctly that those ports do not yet have (published) Qt5 variants/subports.

May I perhaps suggest you to put your files to a place like GitHub or any other public service, so that it becomes slightly easier to track your changes for those who might be interested? Uploading zips to trac starts getting cumbersome at some point, particularly with the amount of changes that you need to make.

Has been the case for months: ​https://github.com/RJVB/macstrop/tree/master/aqua ...

About the install layout/directory structure: I've seen it argued that just putting everything into a single directory is how Digia do it with their own installers, on OS X. True, but the use case they envision is developers who build standalone app bundles that can be submitted to the App Store. That's not the reason for which MacPorts contains Qt, I think. There are Qt ports so there can be any number of other ports to depend on the appropriate Qt version. Most of those dependents probably come from other Unix desktops where Qt does not install in a single directory.

Quick question: do I understand it properly that some of your ports (mostly those based on KDE) do not even work if one installs the complete Qt into a single directory like libexec? If that is the case, we might have a strong argument to try to keep the directories spread out at their usual location. Or do both configurations (either a single directory or different directories) work with most ports?

I do not have time to answer that in detail right now. The quick answer is that I initially used an all-encompassing prefix for Qt4 too when I set out to make qt4-mac concurrent. I remember running into issues, didn't record exactly what, but in retrospect those issues were evidently related to the prefix approach. Basically I consider that MacPorts contains many ports that were conceived for a Freedesktop/XDG environment (including all KDE4 ports), and that as a result we'd do better to stick to something as close to that kind of layout as possible. Which was the case for the "exclusive" qt4-mac layout.

Gotta run...

[pixilla (Bradley Giesbrecht)]

Replying to rjvbertin@…:

Replying to mojca@…:

Replying to rjvbertin@…:

First off, existing users can be warned not to upgrade to 10.11, but I guess you also have newcomers in mind (who'd have to be told to wait until compatibility issues are fixed).

The vast majority of the ones I've installed simply rebuild because 1) they use the Qt4 portgroup and 2) my layout changes are so minimal.

Btw: if we are moving the complete structure anyway, what about renaming the new port simply to qt4? Now that x11 is hardly support (and probably not needed), there is no way to keep calling that port "mac". That way we could keep the old qt4-mac in place for a while (and ports that won't be upgraded yet might conditionally keep working), while urging everyone to switch to the new qt4 asap and update the ports to use qt4 instead of qt4-mac?

I think it would be less confusing for the "standard" qt{4,5} ports to be named qt4 and at5 and let "specialty" ports add a suffix.

Which still boils down to the same thing, at least if I understand correctly that those ports do not yet have (published) Qt5 variants/subports.

May I perhaps suggest you to put your files to a place like GitHub or any other public service, so that it becomes slightly easier to track your changes for those who might be interested? Uploading zips to trac starts getting cumbersome at some point, particularly with the amount of changes that you need to make.

Has been the case for months: ​https://github.com/RJVB/macstrop/tree/master/aqua ...

I hope the substantial time and effort René has put into making QT and KDE better on the Mac can finally make it's way into MacPorts in whole, pieces or spirit. A quick look at the relevant KDE mail lists and issue trackers will show an impressive tail of contributions.

[RJVB (René Bertin)]

Thanks Bradley...!

Replying to mojca@…:

So if I understand properly, installing both ports (your new qt[45]-mac) as well as the transitional subport will make sure that all the ports that haven't been fixed and/or rebuilt yet will happily keep working? And the transitional subport will only be a problem when building other ports against qt5? If that is so, the transition might be a lot less painful than what I imagined.

More or less. I never made a transitional variant or subport for qt5-mac because there were so many (and even fewer *big*) ports that depend on Qt5 and to be honest, because I had none installed. But yes, install my qt4-mac-devel + qt4-mac-devel-transitional, and all your existing Qt4 dependent ports will keep working, and port rev-upgrade won't see a single problem. That was a requirement for me to start working on a Qt4 port, so it's the first thing I made certain worked as it should. It's very simple really; the subport installs symlinks to everything in the new Qt frameworks directory into ${prefix}/Library/Frameworks (and "fake dylibs" into ${prefix}/lib) where those items were installed by the older exclusive port.
Where you're wrong is that the transitional Qt4 subport is no problem for building other ports against Qt5. It's possible that even an equivalent Qt5 transitional subport would NOT clash with the Qt4 transitional port because Qt5's libraries and frameworks have different names

Where things can go awry installing qt4-mac-devel and qt4-mac-devel-transitional is when rev-upgrade starts rebuilding between those 2 installs. That never happened to me, but I always installed them both in a single command, i.e.

port install qt4-mac-devel [+variant(s)] qt4-mac-devel-transitional

I've had some opportunity to double-check what I said earlier, and indeed:

• deactivate qt4-mac-devel-transitional followed by a rev-upgrade to have a break-down of ports not rebuilt against the new concurrent Qt4 install. This also suggest that a rev-bump won't even be necessary.
• reactivate qt4-mac-devel-transitional, and rebuild the listed ports one by one (e.g. with port -ns upgrade --force foo will make them depend on the new install layout

I haven't had time to rebuild all Qt4 dependents that still require rebuilding, but I see no reason to assume that qt4-mac-devel-transitional will still be required afterwards. Except of course to remain compatible with binary packages of Qt4 ports I haven't installed yet.

Potentially the qt4-mac-devel could be called qt4?

It could. Or qt4-devel, if we suppose that it has some kind of development nature.

[michaelld (Michael Dickens)]

Let me be brief: I don't have time to keep up with the ongoing banter here. I'm sure there's lots of great discussion in them, but the best I can do is scan them. I -strongly- encourage replies to be reasonably brief and concise. A reasonable paragraph like mine below is about what I can digest (~250 words). qt4-mac is openmaintainer, so if someone else wants to mess with it please go ahead! If you do, please make BIG changes not very often; as Mojca says: it takes forever to build Qt4 on some systems.

Yes, sometimes it takes a small, stupid thing like total breakage in a new OS to get my attention; I'm a very busy person! I'm going to commit -something- tomorrow/Monday that will install qt4 to allow a parallel build. What I commit will be just the minimum necessary, whatever that is. If we decide to rename "qt4-mac" to just "qt4", that's fine with me & can be done at the same time. If not, that's also fine. I fully agree (and have stated) that René has worked hard & some of his changes will get put in place, some day; maybe by me, maybe by someone else. Maybe we can get a GSoC'16 student to do it & the other accumulating Qt tickets. I think the scenario this coming week will be as Mojca describes: there are just too many ports to check, so let's do the change and then clean up as necessary afterward. My commit will rev-bump around 25 ports that I know work with any Qt4 install location. There are another about 50 ports that either fail (~4 of them) or I haven't been able to test (~46 of them). Let's hope many of the latter of these use the qt4 portgroup & "just work". Either way, the time has come and this is the way it's going to happen (otherwise it won't happen & I think we wall don't want that).

[RJVB (René Bertin)]

I'll try to make this short, but the code extracts cancel most of the effort :)

it takes forever to build Qt4 on some systems.

Interesting, the other day I did the build with nice +15 and only 3 out of my 2*2 cores (2 real, 2 HyperThreads), while watching 2 episodes in QuickTime - and I think the build was done well before I finished the 2nd, i.e. in less than 1.5hours. That's a lot of time to wait turning thumbs, but not exactly "forever" ;) Since it seems more than 1 person here will be rebuilding, maybe we can collect some statistics, for future reference?

Also, I'd be more than happy to maintain a qt4-mac-devel port (i.e. with commit access), for exactly the suggested purpose (port development, test-bed, whatever you want to call it). Getting a GSoC student to handle my changes rather than letting me do it myself ... am I being a jerk if I say that feels like an insult?

qt4-mac vs qt4 "tout court": I'd say there is no real need to change this. Everyone is used to the qt4-mac name for the port, and even if there's no more need to use a -mac vs a -x11 suffix I don't think there's a real advantage to dropping the suffix either. Besides, there's nothing that stops someone from reviving qt4-x11; it should be possible to get it to build still using the latest Qt version that still built for X11 (last time I checked Fink has such a "port" too). Above all, it seems a waste of time to force a rebuild of all dependents just for a change of name ...

The only thing I champion is NOT to use qt4-mac for the prefix path, and that is also because Qt4 locations have been using qt4 directories.

Now, for a minimal change to make Qt4 concurrent, I would propose the following settings in the Qt4 PortGroup (I'm leaving the 2 comments I have in mine):

# standard Qt4 name
global qt_name
set qt_name             qt4
set qt_dir              ${prefix}/libexec/${qt_name}
set qt_dir_rel          libexec/${qt_name}
set qt_docs_dir         ${prefix}/share/doc/${qt_name}
set qt_plugins_dir      ${prefix}/share/${qt_name}/plugins
set qt_mkspecs_dir      ${prefix}/share/${qt_name}/mkspecs
set qt_imports_dir      ${prefix}/share/${qt_name}/imports
set qt_includes_dir     ${prefix}/include/${qt_name}
set qt_libs_dir         ${qt_dir}/lib
set qt_frameworks_dir   ${qt_dir}/Library/Frameworks
set qt_bins_dir         ${qt_dir}/bin
set qt_data_dir         ${prefix}/share/${qt_name}
set qt_translations_dir ${prefix}/share/${qt_name}/translations
set qt_sysconf_dir      ${prefix}/etc/${qt_name}
#set qt_examples_dir     ${prefix}/share/${qt_name}/examples
set qt_examples_dir     ${applications_dir}/Qt4/examples
set qt_demos_dir        ${applications_dir}/Qt4/demos
# no need to change the cmake_module_dir, though I'd have preferred to
# put it into ${prefix}/lib/cmake as qt5-mac also does, and which is the place Linux uses
set qt_cmake_module_dir ${prefix}/share/cmake/Modules
set qt_qmake_cmd        ${qt_dir}/bin/qmake
set qt_moc_cmd          ${qt_dir}/bin/moc
set qt_uic_cmd          ${qt_dir}/bin/uic
set qt_lrelease_cmd     ${qt_dir}/bin/lrelease

• and for configure.args in the Portfile :

# 20151003 : -prefix ${prefix} instead of -prefix ${qt_dir} (as with Qt5) would be
# great, but will lead to frameworks in ${prefix}/Library/Frameworks instead
# of in ${qt_dir}/Library/Frameworks ...
configure.args-append                                     \
    -v                                                    \
    -confirm-license                                      \
    -opensource                                           \
    -prefix          ${qt_dir}                            \
    -bindir          ${qt_bins_dir}                       \
    -libdir          ${qt_libs_dir}                       \
    -docdir          ${qt_docs_dir}                       \
    -headerdir       ${qt_includes_dir}                   \
    -plugindir       ${qt_plugins_dir}                    \
    -importdir       ${qt_imports_dir}                    \
    -datadir         ${qt_data_dir}                       \
    -translationdir  ${qt_translations_dir}               \
    -sysconfdir      ${qt_sysconf_dir}                    \
    -examplesdir     ${qt_examples_dir}                   \
    -demosdir        ${qt_demos_dir}                      \
    -openssl-linked                                       \
    -dbus-linked                                          \
    -fast                                                 \
    -optimized-qmake                                      \
    -no-pch                                               \
    -framework                                            \
    -no-phonon                                            \
    -no-phonon-backend                                    \
    -fontconfig -system-freetype

and to re-absorb the sqlite3 plugin subport into the main port so that port:qt4-mac itself doesn't require that subport in order to be able to run the applications it installs:

# the sqlite3 plugin ("sqlite") has been re-absorbed into the main port however
configure.args-append -system-sqlite
depends_lib-append port:sqlite3
foreach driver {mysql odbc psql sqlite2} {
    configure.args-append -no-sql-${driver}
}

Doing this means removing the port:qt4-mac-sqlite3-plugin dependency in dependent ports, or change them to path:${qt_plugins_dir}/sqldrivers/libqsqlite.dylib:qt4-mac-sqlite3-plugin (as has already been done in the KDE4 PortGroup).

• I'd suggest looking at the changes to build without internal use of exceptions, which is actually the preferred way to build Qt (and the default in Qt5). I've added a variant to re-enable use of exceptions, but I don't think there's any point to that. Contrary to Qt's own -no-exceptions build, mine doesn't disable exceptions in the few components that do require them (just like Qt5 does).

configure.args-append   -no-exceptions

# (26) don't build with exceptions, which gives a completely ABI-compatible build
patchfiles-append       disable-exceptions.patch

post-destroot {
    if {![variant_isset exceptions ]} {
        # building with -no-exceptions will add a section to QtCore/qconfig.h that has to be removed
        # given that we did NOT build QtCore WITHOUT exceptions...
        exec patch -d ${destroot}${qt_frameworks_dir} -Np0 -i ${filespath}/qconfig-remove-EXCEPTIONS.diff
    }
}

IIRC, current port:qt4-mac sets and then "configure.args-delete"'s -optimized qmake, for a reason that's unclear to me. I disabled that and haven't had any issues, and qmake is considerably faster because of it. I also use

pre-configure {
    configure.env-append    MAKEFLAGS=-j${build.jobs}
}

to build qmake in parallel, which can make a big difference. NB: that's the "official" way to do it.

A few useful things from my post-destroot :

post-destroot {
    # Move .apps into the applications_dir, and link each .apps'
    # executable back into ${qt_bins_dir}

    set dr_qt_apps_dir ${destroot}${qt_apps_dir}
    set dr_qt_bins_dir ${destroot}${qt_bins_dir}

    xinstall -m 755 -d ${dr_qt_apps_dir}
    foreach app [glob ${dr_qt_bins_dir}/*.app] {

        # remove the leading stuff
        set app [lindex [split ${app} /] end]

        # move the .app
        move ${dr_qt_bins_dir}/${app} ${dr_qt_apps_dir}
        # link it back
        ln -s ${qt_apps_dir}/${app} ${dr_qt_bins_dir}

        # provide a proxy to the app's executable; symlinks won't
        # be accepted by qtchooser if the user has that port installed.
        set texe [strsed ${app} {g@\.app@@}]
        set appProxy ${dr_qt_bins_dir}/[string tolower ${texe}]
        copy ${filespath}/appProxy.sh ${appProxy}
        reinplace "s|@BUNDLEEXEC@|${qt_apps_dir}/${app}/Contents/MacOS/${texe}|g" ${appProxy}

        ln -s ${qt_qmake_cmd} ${destroot}/${prefix}/bin/qmake-qt${qt_major}
        ln -s ${qt_moc_cmd} ${destroot}/${prefix}/bin/moc-qt${qt_major}
        ln -s ${qt_uic_cmd} ${destroot}/${prefix}/bin/uic-qt${qt_major}
        ln -s ${qt_lrelease_cmd} ${destroot}/${prefix}/bin/lrelease-qt${qt_major}

        # expose KDE4 styles to Qt4:
        ln -s ${prefix}/lib/kde4/plugins/styles ${destroot}${qt_plugins_dir}/
    }
}

That's about all I can think of now. The only thing that I'd really hope to get in at this point are the install locations and to a somewhat lesser extent, preservation of the port name(s) and the re-absorbtion of the sqlite3 plugin. If I'm not overlooking anything else won't introduce incompatibilities with the changes I've been "evolving" and that I'm running locally. I'd just as well like to avoid having to rebuild everything *again* just because "you guys" haven't been keeping up with me :P

[michaelld (Michael Dickens)]

There are some good suggestions in your last comment, René.

• When we get QtChooser and/or qt*-kde implemented, we'll need some of these in place; until then, I think we can leave them alone.

• Moving sqlite3 back to the main qt4-mac makes sense too, because otherwise Assistant fails to execute; either that or moving Assistant to a separate port that depends on the sqlite3 plugin; much easier to do the former.

• I don't know enough about "-no-exceptions" to comment. Can you provide a link that shows that this is the default for Qt-provided binaries? I'm happy to go that way with a variant if it's the usual. At least on Qt4, when I do "./configure --help" I see that "-exceptions" is the default; that's why it is enabled by default.

• No matter the proper way to do a parallel build, MP's way works, yes? If so, I'll leave that as is.

• I think setting "qt_name qt4" is fine. Reads nicely in directory listings too. That's what I use for my internal testing, too.

• my worry about moving data, plugins, and other stuff into ${prefix}/share/qt4 is if/when there is qt4-kde or qt4-x11 that provides its own versions of these. A well-written project will use the specified QMAKE to glean the location of these, so they really don't matter. Hence, I'm inclined to leave them inside the libexec/qt4 area.

I'll look over this list one last time tonight & also review any other comments. Might be tomorrow for the actual checkin, but I'll try for tonight.

[hmeine (Hans Meine)]

AFAIK, -no-exceptions was the default (maybe only de-facto) in the past, in the era of GCC 2.95 and 3.x, when it would seriously speed up large Qt-based projects (e.g. KDE). However, compilers have become more efficient in the meantime, and more people are actually making use of exceptions (e.g. in the past, their use was discouraged in KDE for performance reasons), so the recommendation was changed. (With -no-exceptions, you would have to catch exceptions before entering Qt; I do not want to imply that it is not possible at all to use exceptions with -no-exceptions.)

[RJVB (René Bertin)]

Replying to hans_meine@…:

AFAIK, -no-exceptions was the default (maybe only de-facto) in the past, in the era of GCC 2.95 and 3.x, when it would seriously speed up large Qt-based projects (e.g. KDE). However, compilers have become more efficient in the meantime, and more people are actually making use of exceptions (e.g. in the past, their use was discouraged in KDE for performance reasons), so the recommendation was changed.

I have started implementing this approach after "Qt's own" Thiago Macieira affirmed me that it didn't change anything, and that Qt itself doesn't do anything with exceptions internally except in a few modules. Since those are an integral part of QtCore, all of QtCore is built with exceptions enabled in my approach. I considered going for a finer resolution, but decided that would be overdoing it. My change does take care that the no-exceptions setting does NOT propagate to dependents. The standard way of building Qt4 with -no-exceptions does indeed do that for some reason, but those changes can be reverted (see the post-destroot extract above) and applications using Qt can continue to use exceptions. You can imagine how I discovered that post-destroot bit was required, so yes, I have tested Qt dependent code that uses exceptions.

[RJVB (René Bertin)]

Replying to michaelld@…:

• When we get QtChooser and/or qt*-kde implemented, we'll need some of these in place; until then, I think we can leave them alone.

"These" being what, here? BTW, qtchooser *is* implemented, just not committed: #46029

• Moving sqlite3 back to the main qt4-mac makes sense too, because otherwise Assistant fails to execute; either that or moving Assistant to a separate port that depends on the sqlite3 plugin; much easier to do the former.

Indeed; much easier, and a *lot* faster.

• I don't know enough about "-no-exceptions" to comment. Can you provide a link that shows that this is the default for Qt-provided binaries? I'm happy to go that way with a variant if it's the usual. At least on Qt4, when I do "./configure --help" I see that "-exceptions" is the default; that's why it is enabled by default.

No, I never said it's the default for Qt-provided binaries. With the change I propose Qt4 builds as Qt5 does, without using exceptions internally (except for a few ... exceptions like QtConcurrent) and without support for exceptions enabled in the compiler. That is in fact what we're talking about. As I said in my previous comment, Qt's configure -no-exceptions has effect also on how dependent software is built, but my implementation removes that. With some luck I'll be able to trace the exchange with Thiago in the archives of the interest Qt ML, but that's going to be the only link I can provide. I myself used a variant (first optional, then default) before finally realising that everything just works, and making the -exceptions build under control of a variant.

• No matter the proper way to do a parallel build, MP's way works, yes? If so, I'll leave that as is.

What do you call the MP way, by setting build.jobs? That does NOT work, which is the reason for setting MAKEFLAGS. If you look at the configure script, qmake is built by invoking make without any options at all. I initially patched the configure script, but after asking on the interest ML I was told that "you just have to use MAKEFLAGS=-jn ...

• I think setting "qt_name qt4" is fine. Reads nicely in directory listings too. That's what I use for my internal testing, too.

Yes, much as I think that we should simply maintain the qt4-mac port name, I would hate to see that in the directory structure. Too big, and there is an additional argument against it in the case of Qt5 which I won't repeat here ;)

• my worry about moving data, plugins, and other stuff into ${prefix}/share/qt4 is if/when there is qt4-kde or qt4-x11 that provides its own versions of these.

No, I don't think so. I'm not 100% certain about the plugins of a qt4-x11 build, but qt4-kde will install the exact same things in the shared resource directories as qt4-mac does.

A well-written project will use the specified QMAKE to glean the location of these, so they really don't matter.

The issues I had were not with building. They were runtime issues that IIRC would have required patching the affected dependent ports.

Of course, if you have no objections against a qt4-kde port then all we need to agree upon is how to set up the PortGroup(s) so that users can decide to install either the qt4-mac or qt4-kde flavour (but not both, IMHO) and at most be obliged to support installing from source those ports that do not specifically request the installed Qt4 flavour. You may have seen that I have a draft implementation for Qt5 (converting the Qt5 PortGroup into a "header" that loads either the Qt5-mac or the Qt5-kde PortGroup).

[mojca (Mojca Miklavec)]

Two points:

• Breaking the Qt might be unavoidable, but I would prefer if that would be done just once rather than committing some minimal subset now and then doing another incompatible commit (requiring a revbump of all dependents) with all the desired fixes again one month from now.
• If we would have to support two or more flavours of Qt (for example qt5-mac and qt5-kde) and two variants of all the dependent ports just because qt{4|5}-mac is not configured properly to support KDE applications, that is a very stupid reason to ship two incompatible ports and a great recipe to introduce more confusion. Please let's make sure that qt{4|5} works for everyone (including KDE), so that there won't be any need for "workaround ports".

[RJVB (René Bertin)]

• It's very likely that KF5 will require a Qt5 install with specific patches that we may not want to impose on everyone. Ports cannot depend on a variant, so in that case a dedicated port will be required. I agree that's a much stronger argument for a dedicated port than simply the install layout.

• There is no question of maintaining 2 variants of all dependents. Proper configuration of the PortGroup will ensure that ports that do not have a hard requirement on the one or the other (and thus a conflict with the other or the one) will build with all Qt ports. There could be 1 or 2 corresponding variants that act as a protection against installing the wrong binary build, but that's handled purely in the PortGroup, and maintained by the qt*-kde maintainer (which I reckon would be me).

I've been very clear I think that there is absolutely no reason to change more to the current install layout than strictly necessary, and good reasons to *not* to do that *). It seems Mojca agrees with me; maybe we should invite opinions from more "core" MacPorts developers who are maybe more likely to provide additional arguments based on MacPorts guidelines. ("Follow Linux/Freedesktop/XDG conventions" , "avoid duplicating directory structures", etc.)

I'm compelled to come back one last time to my suggestion to commit my qt4-mac-devel (which wouldn't even "destroy" the existing qt4-mac-devel; that one was very obsolete last time I looked). Mojca voiced concerns about repeated and expensive rebuild cycles, and I understand that we're now facing pressure to "get through with it already" because of 10.11's imminent release.

I cannot really imagine that there will be many early updaters who are on old hardware where building Qt is an issue. Providing an up-to-date qt4-mac-devel will give the other early updaters a working solution that may even be replaced by a drop-in replacement qt4-mac port in the end.

And that wouldn't exclude applying the 10.11 patch to the current port (without even rev-bumping it; not required for users who do not already have an installed version).

*) I do realise that I should have kept a record of the issues I identified while experimenting with an all-encompassing install prefix myself.

[michaelld (Michael Dickens)]

OK; good discussion that I can follow / parse & I'm glad for it. Given the lateness of me getting to this tonight, I'm going to put my proposals here -briefly- for 1 last round of comments & wait to do the formal commits until Tuesday (morning in US/Eastern preferably).

A) Here's what I'm proposing to do tomorrow:

0) commit the minimal patch to allow qt4-mac to install into libexec (just change qt_dir to be "${prefix}/libexec/qt4");
1) commit the proposed (minimal) patch to qt4-mac that allows it to work with 10.11.
These will bring qt4-mac into a state where it works on most target OS X versions but especially with 10.11 as well as allowing for parallel installs. It's not perfect, but it gets things moving & does not require more testing/evaluation. We will have enough work getting Qt4 dependent ports working with the parallel build that I don't think we need more things to deal with just right now.
2) rev-bump ports that are known to work with the updated qt4-mac.

B) I then propose that we create a qt4-mac-devel port that René owns (via his current email) with me and openmaintainer. He can do lots of awesome things with it (Portfile, patches, main repo, whatever), and folks who are brave enough can use it as a drop-in augmented replacement for qt4-mac (yes?). When features reach a state of acceptance / maturity in qt4-mac-devel, they can be ported to qt4-mac en mass; or, if it works well enough, we just use it instead of qt4-mac. In this manner, we get to test drive René's mods while not burdening folks who use qt4-mac & don't really care about all the cool stuff going on under the hood.

C) I further propose that since Qt4 is EOL except for security updates, we just use René Qt4 github repo as the master branch for fixes / patches for qt4-mac-devel. In this way, we can avoid all of the patches that have made qt4-mac so !@#$ difficult to maintain in the past. And, René maintains control over what's in and what's out, with advice from us; and pull requests from us or anybody.

I'm guessing that doing all of the above will actually be a 2-step process. The first step is 0 through 2; I don't want to wait any longer to fix qt4-mac on 10.11, and it makes sense to do the parallel install at the same time. The second step will be doing B and C, and whatever changes are required to qt4-mac to allow it and qt4-mac-devel to be hot-swappable replacements; these will probably require a rev-bump to qt4-mac, but I think that's a fair trade-off for getting functionality in place ASAP then making it robust once we've verified that it all works.

[RJVB (René Bertin)]

Another word about the exceptions "issue": I am using a Qt4 build with these same patches to minimise support for exceptions *inside* Qt on several Linux/KDE4 (Kubuntu 14.04) desktops. I haven't run into a single issue related to building Qt that way.

Replying to michaelld@…:

0) commit the minimal patch to allow qt4-mac to install into libexec (just change qt_dir to be "${prefix}/libexec/qt4");
It's not perfect, but it gets things moving & does not require more testing/evaluation.

The patch is minimal, but you're NOT minimising the amount of work it generates.

We will have enough work getting Qt4 dependent ports working with the parallel build that I don't think we need more things to deal with just right now.

One last time, there will be less work, potentially much less work, by keeping the current install layout like it is as much as possible, and moving only the few parts that are not compatible with a parallel install (*not* "parallel build", that's something else). And if I'm right, all that unnecessary work will have to be undone at some point.

As to shared maintainership on a qt4-mac-devel port: I don't think I like that idea any better than you seem to like the idea of it for port:qt4-mac . With only one of us having commit rights it'll be difficult enough already to maintain the shared part of the qt4 Portgroup.

Or has there been a change in "base" that allows content _resources directories (including the port/group directory) in a "personal" port tree to override the content from the global _resources directory? As long as that's not the case, users of portgroups from my repository have to copy the corresponding files into the global _resources directory.

and whatever changes are required to qt4-mac to allow it and qt4-mac-devel to be hot-swappable replacements;

Hot-swapping after we've basically ended up in the situation of qt5-mac and my proposed qt5-kde, with the former installing all of Qt into a single prefix and the latter moving only the minimal number of components w.r.t. the current layout?

• That will only be possible if my port provides symlinks in ${prefix}/libexec/qt4 that point to the locations where stuff is actually installed.

That's not impossible (I presume), but not exactly any better than the transitional variant I'm providing at the moment.

these will probably require a rev-bump to qt4-mac

As long as those changes do not touch the qt4-mac Portfile but only the PortGroup, a rev-bump shouldn't be required. Not as long as one accepts that users of qt4-mac-devel will have to build every dependent port from source (because the binary builds will all be for qt4-mac users).

[RJVB (René Bertin)]

Can't help myself:

My approach has made the concurrent and "exclusive" qt4-mac ports perfect, drop-in/hot-swappable replacements, meaning I've been able to alternate them via port (de)activate (of *only* the Qt port) which has been helpful a few times.

[michaelld (Michael Dickens)]

Committed in r140960. Now the real work begins, of fixing ports that aren't working with this install location. Fun times ahead!

[michaelld (Michael Dickens)]

I'm updating my local install of qt4-mac, and even without setting MAKEFLAGS all of the CPUs are totally loaded down. I'm guessing that 'port' is using -j4 because my computer has 4 real CPUs and 4 virtual ones (via hyperthreading). So, I'm not convinced that that change is necessary for Mac OS X. It might be for Linux or BSD; I don't know.

[RJVB (René Bertin)]

Replying to michaelld@…:

So, I'm not convinced that that change is necessary for Mac OS X. It might be for Linux or BSD; I don't know.

"That" change being

pre-configure {
    configure.env-append    MAKEFLAGS=-j${build.jobs}
}

is ONLY

to build qmake in parallel

as should be evident from the code.

[RJVB (René Bertin)]

Replying to michaelld@…:

Committed in r140960. Now the real work begins, of fixing ports that aren't working with this install location.

Never too late:

> man porthier

[Polyergic (Shad Sterling)]

Cc Me!

[hmeine (Hans Meine)]

Replying to michaelld@…:

Committed in r140960. Now the real work begins, of fixing ports that aren't working with this install location. Fun times ahead!

In the meantime, I have updated to El Capitan and recompiled (nearly) everything with a fresh MacPorts 2.3.4 installation. However, I am getting Error: Unable to execute port: Can't install qt5-qtbase because conflicting ports are active: qt4-mac which is probably expected behaviour for you, but I assumed that with the latest changes this conflict is no longer present? In other words, what is the official status?

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

After r140960, I have not had any trouble with both qt5 and qt4-mac installed simultaneously.
In qt5, removed conflict notice in r142742.
Any objections to making the same change in qt4-mac?

[michaelld (Michael Dickens)]

I've no objection if others can get qt4 and qt5 installed simultaneously. Go for it!

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Final (hopefully) component finished in r142934.