docbook-xsl @1.79.2: broken man page output

[mouse07410 (Mouse)]

MacOS 10.13.4, Xcode-9.3, current Macports and all the installed ports.

Attempting to generate man pages from XML sources now result in Nroff tags being prefixed with one or more tabs. That in turn causes nroff to not format these pages properly. I just re-installed {libxslt}, and now its own man page is screwed up:

Output of man xsltproc:

XSLTPROC(1)                       xsltproc Manual                      XSLTPROC(1)

NAME
       xsltproc - command line XSLT processor

SYNOPSIS
            .HP 216u

                 xsltproc            [                 [                      | -V
                            | --version                ]                 [
                            | -v                      | --verbose                ]
                       [                      {                           | -o
                                 | --output                     }
                            {                           | FILE
                                 | DIRECTORY                     }
                      ]                 | --timing                 | --repeat
                       | --debug                 | --novalid                 |
       --noout                 | --maxdepth VALUE                 | --html
                       | --encoding ENCODING                  |
       --param PARAMNAME PARAMVALUE                  |
       --stringparam PARAMNAME PARAMVALUE                  | --nonet
                       | --path "PATH(S)"                 | --load-trace
                       | --catalogs                 | --xinclude                 |
       [ | --profile | --norman ]                 | --dumpextensions
                       | --nowrite                 | --nomkdir                 |
       --writesubtree PATH                 | --nodtdattr           ]
       [STYLESHEET]            {                 | XML-FILE...                  |
       -           }

Here's what the beginning of /opt/local/share/man/man1/xsltproc.1 file looks like (observe how .HP tag does not begin at the first column of the line):

$ gunzip -cd /opt/local/share/man/man1/xsltproc.1.gz | head -50
'\" t
.\"     Title: xsltproc
.\"    Author: John Fleck <jfleck@inkstain.net>
.\" Generator: DocBook XSL Stylesheets v1.79.2 <http://docbook.sf.net/>
.\"      Date: $Date$
.\"    Manual: xsltproc Manual
.\"    Source: libxslt
.\"  Language: English
.\"
.TH "XSLTPROC" "1" "$Date$" "libxslt" "xsltproc Manual"
.\" -----------------------------------------------------------------
.\" * Define some portability stuff
.\" -----------------------------------------------------------------
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.\" http://bugs.debian.org/507673
.\" http://lists.gnu.org/archive/html/groff/2009-02/msg00013.html
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\" -----------------------------------------------------------------
.\" * set default formatting
.\" -----------------------------------------------------------------
.\" disable hyphenation
.nh
.\" disable justification (adjust text to left margin only)
.ad l
.\" -----------------------------------------------------------------
.\" * MAIN CONTENT STARTS HERE *
.\" -----------------------------------------------------------------





.SH "NAME"
xsltproc \- command line XSLT processor

.SH "SYNOPSIS"

	.HP \w'\fBxsltproc\fR\ 'u

		\fBxsltproc\fR
		 [
			 [
				 | \fB\-V\fR
				 | \fB\-\-version\fR
			]
			 [
				 | \fB\-v\fR
				 | \fB\-\-verbose\fR

In ​https://github.com/OpenSC/OpenSC.git package every tag in every man page is messed up this way (prefixed by one or more tabs or spaces, so the tag does not start at the beginning of the line, so roff ignores it).

Attaching the page source and the generated result.

Help is appreciated. *Yes, I know this port is not maintained.*

[mouse07410 (Mouse)]

generated man page

[mouse07410 (Mouse)]

XML source

[mouse07410 (Mouse)]

doc/man.xsl:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
	<xsl:import href="docbook.xsl"/>
</xsl:stylesheet

doc/api.css:

body {
  font-family: Verdana, Arial;
  font-size: 0.9em;
}

.title {
  font-size: 1.5em; 
  text-align: center;
}

.toc b {
  font-size: 1.2em;
  border-bottom: dashed 1px black;
}

a {
  color: blue;
  text-decoration: none;
}

a:visited {
  color: blue;
  text-decoration: none;
}

pre.programlisting {
  font-size: 1.1em;
  background-color: #EEEEEE ;
  border: 1px solid #006600 ;
  padding: 1em;
}

span.symbol {
  font-weight: bold;
}

span.errorname {
  font-weight: bold;
}

span.errortext {
  font-style: italic;
}

Example of offending command:

$ xsltproc --debug --nonet --path "./..:/opt/local/share/xsl/docbook-xsl/manpages" --xinclude man.xsl piv-tool.1.xml
Note: namesp. add : added namespace before processing              piv-tool
Warn: meta author : no refentry/info/author                        piv-tool
Note: meta author : see http://www.docbook.org/tdg5/en/html/autho  piv-tool
Warn: meta author : no author data, so inserted a fixme            piv-tool
Note: Writing piv-tool.1
DOCUMENT
version=1.0
encoding=UTF-8
standalone=true
$ head -50 piv-tool.1
'\" t
.\"     Title: piv-tool
.\"    Author: [FIXME: author] [see http://www.docbook.org/tdg5/en/html/author]
.\" Generator: DocBook XSL Stylesheets v1.79.2 <http://docbook.sf.net/>
.\"      Date: 04/25/2018
.\"    Manual: OpenSC Tools
.\"    Source: opensc
.\"  Language: English
.\"
.TH "PIV\-TOOL" "1" "04/25/2018" "opensc" "OpenSC Tools"
.\" -----------------------------------------------------------------
[some lines removed]
.\" -----------------------------------------------------------------
.\" * MAIN CONTENT STARTS HERE *
.\" -----------------------------------------------------------------

	

	.SH "NAME"
piv-tool \- smart card utility for HSPD\-12 PIV cards


	.SH "SYNOPSIS"

		.HP \w'\fBpiv\-tool\fR\ 'u

			\fBpiv\-tool\fR
			 [\fIOPTIONS\fR]

[ryandesign (Ryan Carsten Schmidt)]

I can reproduce this issue on Sierra. The binary archive distributed by our server is not broken, but rebuilding from source with sudo port -ns upgrade --force libxslt causes it to become broken, with the xsltproc manpage misformatted as you describe. The libxslt port hasn't changed materially, so one of its dependencies must have.

Comparing the xsltproc manpage in the binary archive with the one built from source on my system just now, an obvious difference is:

--- before/opt/local/share/man/man1/xsltproc.1	2017-11-03 11:36:26.000000000 -0500
+++ after/opt/local/share/man/man1/xsltproc.1	2018-04-25 10:03:38.000000000 -0500
@@ -1,361 +1,736 @@
+'\" t
 .\"     Title: xsltproc
 .\"    Author: John Fleck <jfleck@inkstain.net>
-.\" Generator: DocBook XSL Stylesheets v1.73.2 <http://docbook.sf.net/>
-.\"      Date: $Date: 2008-04-21 16:28:56 +0200 (Mon, 21 Apr 2008) $
+.\" Generator: DocBook XSL Stylesheets v1.79.2 <http://docbook.sf.net/>
+.\"      Date: $Date$
 .\"    Manual: xsltproc Manual
 .\"    Source: libxslt
+.\"  Language: English
 .\"

The binary archive was built with the system's old docbook-xsl 1.73.2, but even if I just downgrade docbook-xsl to 1.79.1, it builds a working manpage. So the problem was the update of docbook-xsl to 1.79.2 in [04e6959f23bb6b1bd40f40f4d1a3ee2fb43e0714/macports-ports].

[raimue (Rainer Müller)]

I can reproduce the same issue with /usr/bin/xsltproc, so it is definitely caused by docbook-xsl @1.79.2.

[ryandesign (Ryan Carsten Schmidt)]

I filed a bug report: ​https://github.com/docbook/xslt10-stylesheets/issues/65

[mouse07410 (Mouse)]

While the docbook-xsl crowd decides what to do - is it possible to revert the default port on Macports back to 1.78.1?

[ryandesign (Ryan Carsten Schmidt)]

You mean 1.79.1. Yes, it is possible, at the expense of having to add an epoch line to the portfile which can never be removed. Let's wait a short while and see if upstream responds with a fix first. Until then, you can downgrade it on your system: wiki:howto/InstallingOlderPort.

[ryandesign (Ryan Carsten Schmidt)]

In 9230324e4b2d6596870129656c34dc4764ef3e3e/macports-ports (master):

docbook-xsl: Use the non-namespaced version

Fixes rendering of manpages by xsltproc.

Closes: #56354

When the port was updated to 1.79.2 in 04e6959f23bb6b1bd40f40f4d1a3ee2fb43e0714,
it was inadvertently updated to the namespaced version, because upstream
changed the default packaging from non-namespaced to namespaced.

The namespaced version can be found in the separate docbook-xsl-ns port,
which still needs to be updated to the latest version, and ideally
merged into the docbook-xsl port as a subport. Renaming the ports and
updating the dependents might also still be considered.

[ryandesign (Ryan Carsten Schmidt)]

Now, we might still want to try to identify all the ports that were updated between November 5, 2017 and today that use xsltproc to generate their manpages, and increase their revisions so that they get rebuilt properly.

[mouse07410 (Mouse)]

Does it mean that I should be able now to upgrade the docbook-xsl port to 1.79.2 that's in Macports, and it would work?

[ryandesign (Ryan Carsten Schmidt)]

Yes.

[mouse07410 (Mouse)]

Thank you! I confirm it works.

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign:

Now, we might still want to try to identify all the ports that were updated between November 5, 2017 and today that use xsltproc to generate their manpages, and increase their revisions so that they get rebuilt properly.

We also need to include ports that aren't distributable, regardless when they were last updated. And for the benefit of users who always build from source, we should just revbump all ports that build manpages with xsltproc, regardless of when they were updated. But I don't know yet how to construct that list of ports.

[raimue (Rainer Müller)]

1. Get all ports updated between the two commits:

   git diff --name-only 04e6959f23bb6b1bd40f40f4d1a3ee2fb43e0714 9230324e4b2d6596870129656c34dc4764ef3e3e | grep 'Portfile$' | awk -F/ '{print $2}' | tee ports.txt
   

2. Only keep ports with a dependency on docbook-xsl somewhere:

   while read port; do port -q rdeps $port | grep -q docbook-xsl && echo $port; done < ports.txt | tee ports-dep-dbxsl.txt
   

3. Get the archive for each of these ports and check whether it contains at least one page:

   while read port; do sudo port archivefetch $port >/dev/null; a=/opt/local/var/macports/incoming/verified/$port*; tar -q -tf $a opt/local/share/man/ >/dev/null 2>&1 && echo $port; done < ports-dep-dbxsl.txt | tee ports-manpages.txt
   

I did not run these commands to completion as they take time and resources. Maybe it would even be faster to swap steps 2. and 3., as getting all dependencies is so slow compared to just checking a archive for a man page.

[ryandesign (Ryan Carsten Schmidt)]

Thanks, I'll see what I can do with that.

We need to get all ports that use xsltproc, not just those modified between the two commits.

I can use the archives installed on one of the buildbot machines, since using archivefetch won't help with ports that aren't distributable.

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #55946.

[ryandesign (Ryan Carsten Schmidt)]

In c82696955118176c7eb65dff9af70cd7c9ec9533/macports-ports (master):

docbook-xsl: Fix sf backward-compatibility URLs

See: #55253
See: #55255
See: #56354
See: ​https://github.com/docbook/xslt10-stylesheets/issues/65#issuecomment-384467339

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign:

The namespaced version can be found in the separate docbook-xsl-ns port, which still needs to be updated to the latest version, and ideally merged into the docbook-xsl port as a subport. Renaming the ports and updating the dependents might also still be considered.

​https://github.com/macports/macports-ports/pull/1668

[mouse07410 (Mouse)]

This problem re-appeared. Man pages generated from XML are again mis-formatted:

PKCS11-TOOL(1)                                          OpenSC Tools                                          PKCS11-TOOL(1)

     .SH "NAME" pkcs11-tool - utility for managing and using PKCS #11 security tokens

     .SH "SYNOPSIS"

          .HP 288u

               pkcs11-tool                 [OPTIONS]

     .SH "DESCRIPTION"

                     .PP The pkcs11-tool utility is used to manage the data objects on smart cards and similar PKCS #11
security tokens. Users can list and read PINs, keys and certificates stored on the token. User PIN authentication is
performed for those operations that require it.

     .SH "OPTIONS"

                     .PP

Installed versions:

$ port installed | grep docbook
  docbook-xml @5.0_3 (active)
  docbook-xml-4.1.2 @5.0_1 (active)
  docbook-xml-4.2 @5.0_1 (active)
  docbook-xml-4.3 @5.0_1 (active)
  docbook-xml-4.4 @5.0_1 (active)
  docbook-xml-4.5 @5.0_1 (active)
  docbook-xml-5.0 @5.0_1 (active)
  docbook-xsl-nons @1.79.2_0 (active)

Help please!

[ryandesign (Ryan Carsten Schmidt)]

I'm not able to reproduce that problem. pkcs11-tool is provided by the opensc port, and when I install the opensc port, the pkcs11-tool(1) manpage is properly formatted, regardless of whether I build opensc from source or get a binary from our server.

Make sure your opensc port is up to date, that you have installed opensc @0.18.0_1. If you have opensc @0.18.0_0 or earlier, upgrade it with sudo port upgrade opensc. If you do still see the problem with opensc @0.18.0_1, rebuild it with sudo port -n upgrade --force opensc.

[mouse07410 (Mouse)]

I'm building OpenSC from the master ​https://github.com/OpenSC/OpenSC.git or ​https://github.com/mouse07410/OpenSC.git. Last time this worked OK was May 2, 2018. On one computer the pages were not rebuilt since, and are OK. On the other one they were rebuilt today, and about half (or more) or them are broken.

Could you please try to build OpenSC from the (upstream) master, and let me know?

Thanks!

[mouse07410 (Mouse)]

This is what the "good" man page heading looks like:

'\" t
.\"     Title: pkcs11-tool
.\"    Author: [FIXME: author] [see http://www.docbook.org/tdg5/en/html/author]
.\" Generator: DocBook XSL Stylesheets vsnapshot <http://docbook.sf.net/>
.\"      Date: 05/02/2018
.\"    Manual: OpenSC Tools
.\"    Source: opensc
.\"  Language: English
.\"
.TH "PKCS11\-TOOL" "1" "05/02/2018" "opensc" "OpenSC Tools"
.\" -----------------------------------------------------------------
.\" * Define some portability stuff
.\" -----------------------------------------------------------------
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.\" http://bugs.debian.org/507673
.\" http://lists.gnu.org/archive/html/groff/2009-02/msg00013.html
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[ryandesign (Ryan Carsten Schmidt)]

I checked out the opensc master, and when I ran ./configure with no arguments, the output included the line

checking xsl-stylesheets... no

and no manpages were generated. When I added the configure argument --with-xsl-stylesheetsdir=/opt/local/share/xsl/docbook-xsl-nons, as the Portfile does, the stylesheets were found, and the pkcs11-tool manpage was generated correctly. So, I don't think the problem you are now experiencing is a bug in MacPorts.

[mouse07410 (Mouse)]

Thank you! Let me try that option (I did not use it before).

Would you be kind enough to try the same process with ​https://github.com/mouse07410/OpenSC.git? I'd really appreciate it, if you could.

Thanks!

[ryandesign (Ryan Carsten Schmidt)]

Replying to mouse07410:

Would you be kind enough to try the same process with ​https://github.com/mouse07410/OpenSC.git? I'd really appreciate it, if you could.

The result is the same with your fork.

[mouse07410 (Mouse)]

Thank you!!

What can be wrong with what I'm doing? I've added your argument, so the relevant part of MacOSX/build-package.in looks like:

if ! test -e ${BUILDPATH}/target/$PREFIX/lib/pkgconfig; then
        ./configure --prefix=$PREFIX \
                --sysconfdir=$PREFIX/etc \
                --enable-cvcdir=$PREFIX/etc/cvc \
                --enable-x509dir=$PREFIX/etc/x509 \
                --disable-dependency-tracking \
                --enable-shared \
                --enable-static \
                --enable-strict \
                --disable-assert \
                --with-xsl-stylesheetsdir=/opt/local/share/xsl/docbook-xsl-nons \
                --enable-sm # TODO: remove this (must be sensible default in master)

        # always make clean
        make clean

        # compile
        make -j 2

        # copy files
        rm -rf ${BUILDPATH}/target
        make install DESTDIR=${BUILDPATH}/target

and yet the docs/man pages aren't rebuilt. The binaries are recompiled/relinked...

[mouse07410 (Mouse)]

Forced rebuild on another machine - and with your fix it worked correctly. Man page that was mis-formatted before, displayed correctly.

Thank you!

[mouse07410 (Mouse)]

Oh, and please feel free to close this one (again ;).

[mouse07410 (Mouse)]

And thank you for your help, especially with the extra testing on your systems.