shared-mime-info @2.0 does not build on PPC Leopard, Mac OS X 10.5.8, because http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd could no be loaded

[ballapete (Peter "Pete" Dyballa)]

Executing:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/build" && /opt/local/bin/ninja -v 
DEBUG: system:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/build" && /opt/local/bin/ninja -v 
[1/8] /opt/local/bin/xmlto -o data/shared-mime-info-spec-html html-nochunks ../shared-mime-info-2.0/data/shared-mime-info-spec.xml
FAILED: data/shared-mime-info-spec-html 
/opt/local/bin/xmlto -o data/shared-mime-info-spec-html html-nochunks ../shared-mime-info-2.0/data/shared-mime-info-spec.xml
xmlto: /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/build/../shared-mime-info-2.0/data/shared-mime-info-spec.xml does not validate (status 3)
xmlto: Fix document syntax or use --skip-validation option
I/O error : Attempt to load network entity http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd
/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/build/../shared-mime-info-2.0/data/shared-mime-info-spec.xml:6: warning: failed to load external entity "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"
]>
  ^
I/O error : Attempt to load network entity http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd
warning: failed to load external entity "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"
validity error : Could not load the external subset "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"
Document /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/build/../shared-mime-info-2.0/data/shared-mime-info-spec.xml does not validate
[2/8] /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/shared-mime-info-2.0/data/freedesktop_generate.sh /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/shared-mime-info-2.0 /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/build
ninja: Entering directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/build'
[0/1] Running external command shared-mime-info-gmo
ninja: build stopped: subcommand failed.
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/build" && /opt/local/bin/ninja -v 
Exit code: 1

[ballapete (Peter "Pete" Dyballa)]

Main.log from PPC Leopard

[ballapete (Peter "Pete" Dyballa)]

Astonishingly it built without any problem on PPC Tiger, Mac OS X 10.4.11…

[kencu (Ken)]

built on 10.5 PPC for me.

$ port -v installed shared-mime-info
The following ports are currently installed:
  shared-mime-info @2.0_0 (active) platform='darwin 9' archs='ppc' date='2020-06-22T21:48:04-0700'

inconsistent building on different machines sounds like a possible race condition with parallel builds, but not sure.

[ballapete (Peter "Pete" Dyballa)]

Replying to kencu:

inconsistent building on different machines sounds like a possible race condition with parallel builds, but not sure.

Single core single CPU 7447A – not likely.

[kencu (Ken)]

well, I guess something else then.

[michaelld (Michael Dickens)]

I just updated:

$ port -v installed shared-mime-info
The following ports are currently installed:
  shared-mime-info @2.0_0 (active) platform='darwin 9' archs='ppc' date='2020-06-25T16:55:33-0400'

This is with an iMac G5 @ 1.6 GHz (PowerMac8,1).

[michaelld (Michael Dickens)]

It has something to do with networking issue; here's the issue(s) ... which do not appear in my version's build log (just the "[1/8]" line):

:info:build [1/8] /opt/local/bin/xmlto -o data/shared-mime-info-spec-html html-nochunks ../shared-mime-info-2.0/data/shared-mime-info-spec.xml
:info:build FAILED: data/shared-mime-info-spec-html 
:info:build /opt/local/bin/xmlto -o data/shared-mime-info-spec-html html-nochunks ../shared-mime-info-2.0/data/shared-mime-info-spec.xml
:info:build xmlto: /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/build/../shared-mime-info-2.0/data/shared-mime-info-spec.xml does not validate (status 3)
:info:build xmlto: Fix document syntax or use --skip-validation option
:info:build I/O error : Attempt to load network entity http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd
:info:build /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/build/../shared-mime-info-2.0/data/shared-mime-info-spec.xml:6: warning: failed to load external entity "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"
:info:build ]>
:info:build   ^
:info:build I/O error : Attempt to load network entity http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd
:info:build warning: failed to load external entity "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"
:info:build validity error : Could not load the external subset "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"
:info:build Document /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_shared-mime-info/shared-mime-info/work/build/../shared-mime-info-2.0/data/shared-mime-info-spec.xml does not validate

[kencu (Ken)]

does xmlto download stuff? I saw that line too, but could not sort out what was happening there.

[ballapete (Peter "Pete" Dyballa)]

Replying to michaelld:

I tried to upgrade twice. In-between were hours (and also a port selfupdate). First time it was night in Europe, second time it was before noon. I saw this line(s), thinking of fake report – why should something try a download when the many docbook ports are all installed?

[ryandesign (Ryan Carsten Schmidt)]

The file shared-mime-info-2.0/data/shared-mime-info-spec.xml specifies that its ​dtd is -//OASIS//DTD DocBook XML V4.1.2//EN and that it can be downloaded from ​http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd. Tools like xmlto need the dtd to work with the document, and if a local copy of that dtd isn't installed they have to download the dtd. That seems to be what's happening in your case for an unknown reason, and the download is failing for another unknown reason.

You're right that there shouldn't need to be any network download happening here, because the docbook-xml-4.1.2 port should have already provided a copy of this dtd and it should be in your catalog. Can you check if it is? What's the contents of /opt/local/etc/xml/catalog?

[RJVB (René Bertin)]

Replying to ryandesign:

You're right that there shouldn't need to be any network download happening here

Am I confounding with other build bots (launchpad) or do the MacPorts ones also block internet access during the build phase? If so and if the resource cannot be relied upon to be installed then a fallback should be provided by the port.

[ballapete (Peter "Pete" Dyballa)]

Replying to ryandesign:

You're right that there shouldn't need to be any network download happening here, because the docbook-xml-4.1.2 port should have already provided a copy of this dtd and it should be in your catalog. Can you check if it is? What's the contents of /opt/local/etc/xml/catalog?

Ahh! It's missing a particular entry:

<?xml version="1.0"?>
<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN"
    "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">

<!-- Created by XML Catalog Manager 2.2 -->
<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">
<nextCatalog catalog="/opt/local/share/xml/docbook/4.3/catalog.xml" />
<nextCatalog catalog="/opt/local/share/xml/docbook/4.4/catalog.xml" />
<nextCatalog catalog="/opt/local/share/xml/docbook/4.5/catalog.xml" />
<nextCatalog catalog="/opt/local/share/xml/docbook/4.2/catalog.xml" />
<nextCatalog catalog="/opt/local/share/xml/docbook/5.0/catalog.xml" />
<nextCatalog catalog="/opt/local/share/xsl/docbook-xsl-nons/catalog.xml" />
<nextCatalog catalog="/opt/local/share/xsl/docbook-xsl-nons/catalog.sf.xml" />
<nextCatalog catalog="/opt/local/share/xsl/docbook-xsl-ns/catalog.xml" />
<nextCatalog catalog="/opt/local/share/xsl/docbook-xsl-ns/catalog.sf.xml" />
</catalog>

I'll update manually and check with Tiger's version!

[ballapete (Peter "Pete" Dyballa)]

Replying to ballapete:

Replying to ryandesign:

I'll update manually and check with Tiger's version!

It built now instantly!

[ryandesign (Ryan Carsten Schmidt)]

Replying to RJVB:

Am I confounding with other build bots (launchpad) or do the MacPorts ones also block internet access during the build phase?

The MacPorts buildbot infrastructure at macOS forge was under strict networking restrictions and we did see build failures for ports that tried to download things at build time. But the new buildbot infrastructure that I have hosted since end of 2016 does not have those restrictions.

[ryandesign (Ryan Carsten Schmidt)]

Replying to ballapete:

Ahh! It's missing a particular entry:

Ok so we should figure out what went wrong.

Here's how it should work:

When you run sudo port -f deactivate docbook-xml-4.1.2 the line <nextCatalog catalog="/opt/local/share/xml/docbook/4.1.2/catalog.xml" /> should be removed from /opt/local/etc/xml/catalog.

When you run sudo port activate docbook-xml-4.1.2 that line should be added to the catalog again.

Does that happen?

If not, what about for other docbook-xml ports?

[ballapete (Peter "Pete" Dyballa)]

Replying to ryandesign:

Replying to ballapete:

Ahh! It's missing a particular entry:

When you run sudo port -f deactivate docbook-xml-4.1.2 the line <nextCatalog catalog="/opt/local/share/xml/docbook/4.1.2/catalog.xml" /> should be removed from /opt/local/etc/xml/catalog.

Not that perfectly… First deactivation failed, but entry was removed. Repeating deactivation and reactivation it worked smoothly:

root 232 /\ port -f deactivate docbook-xml-4.1.2
--->  Unable to deactivate docbook-xml-4.1.2 @5.0_1, the following ports depend on it:
--->  	docbook-xml @5.0_3
Warning: Deactivate forced.  Proceeding despite dependencies.
--->  Deactivating docbook-xml-4.1.2 @5.0_1
Error: Failed to deactivate docbook-xml-4.1.2: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_registry_portfiles_docbook-xml-4.1.2-5.0_1_b4f935313d25f499bfb63c58dbc0c2faba45a55d0d4b733381c776a7be9fecaa-7399/docbook-xml-4.1.2/main.log for details.
Warning: Failed to execute portfile from registry for docbook-xml-4.1.2 @5.0_1
root 233 /\ l /opt/local/etc/xml/catalog
-rw-r--r--  1 root  admin  956 26 Jun 18:39 /opt/local/etc/xml/catalog
root 234 /\ grep 4.1.2 /opt/local/etc/xml/catalog
root 235 /\ port activate docbook-xml-4.1.2
--->  Computing dependencies for docbook-xml-4.1.2
--->  Activating docbook-xml-4.1.2 @5.0_1
--->  Cleaning docbook-xml-4.1.2
root 236 /\ grep 4.1.2 /opt/local/etc/xml/catalog
<nextCatalog catalog="/opt/local/share/xml/docbook/4.1.2/catalog.xml" />
root 237 /\ port -f deactivate docbook-xml-4.1.2
--->  Unable to deactivate docbook-xml-4.1.2 @5.0_1, the following ports depend on it:
--->  	docbook-xml @5.0_3
Warning: Deactivate forced.  Proceeding despite dependencies.
--->  Deactivating docbook-xml-4.1.2 @5.0_1
--->  Cleaning docbook-xml-4.1.2
root 238 /\ port activate docbook-xml-4.1.2
--->  Computing dependencies for docbook-xml-4.1.2
--->  Activating docbook-xml-4.1.2 @5.0_1
--->  Cleaning docbook-xml-4.1.2
root 239 /\ grep 4.1.2 /opt/local/etc/xml/catalog
<nextCatalog catalog="/opt/local/share/xml/docbook/4.1.2/catalog.xml" />

If not, what about for other docbook-xml ports?

The other versions just worked to deactivate and reactivate. Did not look into that main.log file…

[ryandesign (Ryan Carsten Schmidt)]

Replying to ballapete:

Error: See /opt/local/var/macports/logs/_opt_local_var_macports_registry_portfiles_docbook-xml-4.1.2-5.0_1_b4f935313d25f499bfb63c58dbc0c2faba45a55d0d4b733381c776a7be9fecaa-7399/docbook-xml-4.1.2/main.log for details.

Sounds like it's working now... Do you still have this main.log file? If so, its contents might tell us what went wrong before.

[RJVB (René Bertin)]

Maybe nothing actually went wrong, maybe the entry for this particular (and possibly other?) file(s) had disappeared in a perfectly explicable way.

One such explanation I could think of is if you install other software (from) source, let's say into /usr/local, but using libraries from $prefix (which I presume is a valid use case). Software that installs docbook files will probably invoke the indexer to update the catalog, but couldn't that lead to a catalog that indexes only the docbook files installed under /usr/local?

[ballapete (Peter "Pete" Dyballa)]

Replying to ryandesign:

Replying to ballapete:

Sounds like it's working now... Do you still have this main.log file? If so, its contents might tell us what went wrong before.

No, it's lost. Upon activating shared-mime-info again port cleaned everything… (I checked that file afterwards, but it was already gone.)

[ballapete (Peter "Pete" Dyballa)]

Replying to RJVB:

One such explanation I could think of is if you install other software (from) source, let's say into /usr/local, but using libraries from $prefix (which I presume is a valid use case). Software that installs docbook files will probably invoke the indexer to update the catalog, but couldn't that lead to a catalog that indexes only the docbook files installed under /usr/local?

No other installation exists, the 4.1.2 version just was not recorded.

[ryandesign (Ryan Carsten Schmidt)]

Ok, then we'll just call it a mystery. If the problem recurs and you can provide a log or more information like a reproduction recipe, let us know.