shared-mime-info @1.3 update-mime-database output is not removed on deactivate

[bgilbert (Benjamin Gilbert)]

shared-mime-info creates the MIME database in its post-activate hook but does not delete it on deactivate. Thus, when shared-mime-info is uninstalled, applications continue to behave as though it is still present. That is confusing, and certainly not what the user intended by deactivating the package.

For reference, here is ​the postrm script from Debian's package.

[dbevans (David B. Evans)]

Will handle this as part of the resolution of #45354. Thanks.

[ctreleaven (Craig Treleaven)]

Note that shared-mime-info has also been updated to v. 1.4, as well.

[ryandesign (Ryan Carsten Schmidt)]

Replying to dbevans:

Will handle this as part of the resolution of #45354. Thanks.

It didn't get fixed as part of #45354.

[mascguy (Christopher Nielsen)]

Reassigning to current maintainer

[RJVB (René Bertin)]

This isn't exactly the only port that generates files in the post-activate which are not cleaned up when the port is deactivated. As a random example, from port:meld: ` post-activate {

system "${prefix}/bin/update-desktop-database ${prefix}/share/applications" system "${prefix}/bin/gtk-update-icon-cache-3.0 -f -t ${prefix}/share/icons/hicolor" system "${prefix}/bin/update-mime-database ${prefix}/share/mime" system "${prefix}/bin/glib-compile-schemas ${prefix}/share/glib-2.0/schemas"

} `

This has never caused much eye-brow-raising that I know of. The history of this ticket underlines how much of a non-issue it is in practice.

I'd suggest that a mechanism is added through which ports can register which files it (might) create (be it in post-activate or otherwise) that should ideally be removed when the port is deactivated or uninstalled. That would save a lot of duplicating of trivial code.

I will point out however that in my experience the post-activate step is not (reliably) performed when re-activating a port that had been deactivated, only after an install/upgrade.

[mascguy (Christopher Nielsen)]

Here's the thing: We have a mountain of old neglected tickets like this. And I've spent a tremendous amount of time digging through them over the past few months.

So if you could please close this issue - either by fixing (my vote), or rejecting as 'wontfix' - I'd really appreciate it!

[RJVB (René Bertin)]

I'll look into that then, when I get back to my Mac; don't hesitate to ping me about it in 2 weeks or so because I'm likely to let this slip to the back of my mind.

It just struck me that we could probably use launchd to monitor the directory and have update-mime-database be called automatically when things change in there. That saves having to perform the update in portfiles at activation and deactivation (which only 3 ports appear to do, currently).

[RJVB (René Bertin)]

So... pre-activate is indeed not reliable for me, it seems need a preceding install event. But post-deactivate does work every time.

That means there will need to be a launchd agent or daemon similar to what KDE4 does with kbuildsycoca4, which watches $prefix/share/mime/packages . Without that, activating a different version of shared-mime-database would leave you without the cached info.

The question now is what has to be deleted. Obviously I cannot just remove all of $prefix/share/mime, but is there anything other than $prefix/share/mime/packages that needs to remain?

[RJVB (René Bertin)]

Replying to RJVB:

The question now is what has to be deleted. Obviously I cannot just remove all of $prefix/share/mime, but is there anything other than $prefix/share/mime/packages that needs to remain?

Does this look correct?

post-deactivate {
    # cleanup
    foreach f [glob -nocomplain ${prefix}/share/mime/*] {
        if {${f} ne "${prefix}/share/mime/packages"} {
            ui_debug "file delete -force ${f}"
        }
    }
}

[mascguy (Christopher Nielsen)]

Replying to RJVB:

Does this look correct?

post-deactivate {
    # cleanup
    foreach f [glob -nocomplain ${prefix}/share/mime/*] {
        if {${f} ne "${prefix}/share/mime/packages"} {
            ui_debug "file delete -force ${f}"
        }
    }
}

Since I don't have the tribal knowledge that our veteran members do, I'd like others to chime in as well. But based on a naive understanding of all of this - and some quick validation - that looks reasonable.

Semi-confirmed via the following, which suggests your approach looks promising:

$ gfind /opt/local/share/mime -type f -print0 | gxargs -0 -r port provides | ggrep --invert-match "not provided by"
/opt/local/share/mime/packages/avif.xml is provided by: libheif
/opt/local/share/mime/packages/gcr-crypto-types.xml is provided by: gcr
/opt/local/share/mime/packages/bluefish.xml is provided by: bluefish
/opt/local/share/mime/packages/heif.xml is provided by: libheif
/opt/local/share/mime/packages/glom.xml is provided by: glom
/opt/local/share/mime/packages/freedesktop.org.xml is provided by: shared-mime-info
/opt/local/share/mime/packages/gramps.xml is provided by: gramps
/opt/local/share/mime/packages/org.bunkus.mkvtoolnix-gui.xml is provided by: mkvtoolnix-devel
/opt/local/share/mime/packages/anjuta.xml is provided by: anjuta

Anyone else, thoughts...?

[mascguy (Christopher Nielsen)]

Technically, the safest approach might be to actually check each file before deletion, to see whether it was explicitly installed by another port. Perhaps that's overkill though...?

[RJVB (René Bertin)]

Replying to mascguy:

$ gfind /opt/local/share/mime -type f -print0 | gxargs -0 -r port provides

;) You know you can just do

port provides `find /opt/local/share/mime -depth -type f`

I just tried the equivalent on Linux:

> apt contains /usr/share/mime/*
dpkg-query: no path found matching pattern /usr/share/mime/aliases
dpkg-query: no path found matching pattern /usr/share/mime/all
dpkg-query: no path found matching pattern /usr/share/mime/application
dpkg-query: no path found matching pattern /usr/share/mime/audio
dpkg-query: no path found matching pattern /usr/share/mime/generic-icons
dpkg-query: no path found matching pattern /usr/share/mime/globs
dpkg-query: no path found matching pattern /usr/share/mime/globs2
dpkg-query: no path found matching pattern /usr/share/mime/icons
dpkg-query: no path found matching pattern /usr/share/mime/image
dpkg-query: no path found matching pattern /usr/share/mime/inode
dpkg-query: no path found matching pattern /usr/share/mime/magic
dpkg-query: no path found matching pattern /usr/share/mime/message
dpkg-query: no path found matching pattern /usr/share/mime/mime.cache
dpkg-query: no path found matching pattern /usr/share/mime/model
dpkg-query: no path found matching pattern /usr/share/mime/multipart
libreoffice6.1-debian-menus, skrooge, shared-mime-info, scilab-cli, qtiplot, plasma-widget-lancelot, okteta, nautilus-data, massif-visualizer, code, virtualbox-6.1, packagekit, libakonadi-socialutils4, libakonadi-kde4, tracktion7, krita-data, kmymoney-common, kdevelop-data, kdepim-runtime, kdelibs5-data, kde-runtime-data, k3b-data, ib2014, hugin, gnash-common, gcr, fontforge, docutils-common, click, calligraplan, calligra-data, brasero-common, bluedevil, akonadi-server, zoom: /usr/share/mime/packages
dpkg-query: no path found matching pattern /usr/share/mime/subclasses
dpkg-query: no path found matching pattern /usr/share/mime/text
dpkg-query: no path found matching pattern /usr/share/mime/treemagic
dpkg-query: no path found matching pattern /usr/share/mime/types
dpkg-query: no path found matching pattern /usr/share/mime/uri
dpkg-query: no path found matching pattern /usr/share/mime/version
dpkg-query: no path found matching pattern /usr/share/mime/video
dpkg-query: no path found matching pattern /usr/share/mime/x-content
dpkg-query: no path found matching pattern /usr/share/mime/x-epoc
dpkg-query: no path found matching pattern /usr/share/mime/XMLnamespaces

That seems to confirm my hypothesis, and also suggests it'd be overkill to check each individual file. Esp. since that would require writing a directory scanner. Not that I've never done that in Tcl but it's a bit wordy.

What I could do is invoke port -v rev-upgrade | fgrep ${prefix}/share/mime in the post-deactivate, which should give a list of any missing files ;)

[mascguy (Christopher Nielsen)]

Barring any thoughts/objections from others, your approach looks reasonable. Let's go with it, and close this ticket!

[RJVB (René Bertin)]

I'll do a PR this weekend because there's another thing I'd like to propose ... and I'd like to hear about how the proposed changes work for others.

[RJVB (René Bertin)]

In 03217ed036984ba9a5be994bf780e88d819fad1f/macports-ports (master):

shared-mime-info: clean up cache on deactivate

• Also replace the post-activate call to update-mime-database with a launchd daemon watching ${prefix}/share/mime/packages

Closes: #45396

[mascguy (Christopher Nielsen)]

Adding Clemens to the mix

[RJVB (René Bertin)]

Anyway, I forgot to remove myself as a maintainer, can you please add that to the post-PR cleanup?

[neverpanic (Clemens Lang)]

As outlined in ​https://trac.macports.org/ticket/67533#comment:19, I really don't like the use of autostarting things for this. We should not auto-start anything on users systems without an action from them that tells us this is OK. In the few ports that use autostart so far, installing the port is this step (because the ports aren't dependencies of anything that get auto-installed).

I vaguely recall that other linux distributions use triggers to handle this sort of problem, i.e., any package that installs a file in a certain path eventually triggers an update of the cache database at the end of the operation. Maybe it's time to add such a mechanism to MacPorts? On the other hand, shouldn't ports that install files in $prefix/share/mime/packages already include an invocation of update-mime-database in their post-activation scripts to work correctly? Is that commonly not the case (and if it is, why isn't this a big problem already)?

If you're doing this change just to delete a few files on deactivation of the shared-mime-info port, that can be done with Tcl in a pre-deactivate phase and with some magic to use the interfaces to query the registry. It's entirely doable to check whether a given file was installed by a port from Tcl in a Portfile environment.

[mascguy (Christopher Nielsen)]

Replying to neverpanic:

I vaguely recall that other linux distributions use triggers to handle this sort of problem, i.e., any package that installs a file in a certain path eventually triggers an update of the cache database at the end of the operation. Maybe it's time to add such a mechanism to MacPorts? On the other hand, shouldn't ports that install files in $prefix/share/mime/packages already include an invocation of update-mime-database in their post-activation scripts to work correctly? Is that commonly not the case (and if it is, why isn't this a big problem already)?

Yep, if base had such support, that would be great! And it would eliminate any need to implement our own custom filewatcher/daemon to catch updates to the MIME area.

If you're doing this change just to delete a few files on deactivation of the shared-mime-info port, that can be done with Tcl in a pre-deactivate phase and with some magic to use the interfaces to query the registry. It's entirely doable to check whether a given file was installed by a port from Tcl in a Portfile environment.

That's not a problem. The challenge is to monitor the MIME area, and update the database when things are added/removed by *other* ports.

[Christopher Nielsen <mascguy@…>]

In b86863ea623b08d65e11d7fd28b05b07e503d593/macports-ports (master):

shared-mime-info: drop ownership for @rjvb, as requested
See: #45396

[mascguy (Christopher Nielsen)]

Replying to neverpanic:

Replying to mascguy:

That's the reason override settings like startupitem_autostart exist in macports.conf. Set those as you wish, and you're done.

I think this is making things too simple on ourselves. Most users won't know that the setting exists, and only a few users will even bother to ask. Other users will be appalled by the decision to auto-start something on their machines, and silently leave, patch locally, or rant about it on social media. Our users should never even be in the situation where they need to consider setting startupitem_autostart no in the first place. As a project contributor, I should have no incentive to set this option, and yet here we are and I just went and changed this setting on my system to prevent this from running automatically without my approval.

One thought on this, albeit from a broader perspective: For configuration settings that are more likely to be changed, relatively benign, etc, perhaps we might want to consider providing a friendlier front-end to allow the user to both a) list them; and b) change them.

Whether startupitem_autostart falls under the category of "things we'd like to expose and let users change," is another story.

Regardless, it's the same idea as git config xxx: Let users see what's setup - at least for a few key ones - and let them update if they choose. (In our case, perhaps the verb might be port prefs, to avoid confusion/clashes with port configure.)

Presumably you folks have already discussed this years ago, and likely already submitted a Trac enhancement request(s). Nonetheless, it's worth mentioning!

Your folks' thoughts?

[RJVB (René Bertin)]

I'll offer another suggestion. Debuntu as a "suggested packages" option where a package can suggest to install related packages. When a port needs an automatic updater service to function correctly it could provide a subport which installs and activates that service. As with Debuntu such related ports would be installed automatically under normal circumstances, unless the user indicates otherwise (on the commandline for this particular install, or by disabling the feature alltogether).

From there to adding a weight to the suggestion is only a small step.

[mascguy (Christopher Nielsen)]

There's one minor thing we missed, when merging the new daemon-based automatic refresh: Ports that explicitly try to update the MIME db during post-activate, fail (depending on the exact timing of when the former runs):

    Failed to rename /opt/local/share/mime/inode/socket.xml.new as /opt/local/share/mime/inode/socket.xml: No such file or directory
    Command failed: /opt/local/bin/update-mime-database /opt/local/share/mime
    Exit code: 1

But rather than modify the numerous ports that have that already in place, it would be easier to simply make the existing binary a stub (with the real one renamed).

I'll tackle that today, if possible.

[RJVB (René Bertin)]

I count less than 30 ports which call update-mime-database so removing the few lines isn't that enormous a task.

Rather than making update-mime-database a stub, make it touch $prefix/share/mime/packages/freedesktop.org.xml or maybe even /opt/local/share/mime/packages (cannot test right now if that would also trigger the updater). That way the command retains its effect but should no longer be able to cause a race condition.

[mascguy (Christopher Nielsen)]

Replying to RJVB:

I count less than 30 ports which call update-mime-database so removing the few lines isn't that enormous a task.

True, but I'd rather keep those in place, in case we follow a different approach long-term.

Rather than making update-mime-database a stub, make it touch $prefix/share/mime/packages/freedesktop.org.xml or maybe even /opt/local/share/mime/packages (cannot test right now if that would also trigger the updater). That way the command retains its effect but should no longer be able to cause a race condition.

Yep, agreed... that would be a better option.

[Christopher Nielsen <mascguy@…>]

In 76feeed12d8d08319c82eaf36dab9fa7512fc9a3/macports-ports (master):

shared-mime-info: make update-mime-database a stub, using touch

• Allows ports to use update-mime-database, without interfering with daemon
• Real updater script now named update-mime-database-real

Closes: #45396

[Christopher Nielsen <mascguy@…>]

In 95999aeec69571c65ecd14c1700bfe5d1226762a/macports-ports (master):

shared-mime-info: for non-darwin, symlink stub to real

See: #45396

[Christopher Nielsen <mascguy@…>]

In 5434013047a9976ccf74a846d888bfdf5afb5bba/macports-ports (master):

shared-mime-info: stub script: add path checks

• For ports like 'synfigstudio', which run 'update-mime-database' for a destroot path, ensure we don't fail.

See: #45396
Fixes: #68188