[PATCH 0/7] Macro for Asciidoctor support

There are two major processors of AsciiDoc: AsciiDoc itself, and Asciidoctor.
Both have advantages and disadvantages, but traditionally the documentation has
been built with AsciiDoc, leading to some surprising breakage when building with
Asciidoctor.  Partially, this is due to the need to specify a significant number
of macros on the command line when building with Asciidoctor.

This series cleans up some issues building the documentation with Asciidoctor
and provides two knobs, USE_ASCIIDOCTOR, which controls building with
Asciidoctor, and ASCIIDOCTOR_EXTENSIONS_LAB, which controls the location of the
Asciidoctor Extensions Lab, which is necessary to expand the linkgit macro.

The need for the extensions could be replaced with a small amount of Ruby code,
if that's considered desirable.  Previous opinions on doing so were negative,
however.

In the process, I found several issues with cat-texi.perl, which have been
fixed.  It has also been modernized to use strict, warnings, and lexical file
handles.  I also made an attempt to produce more diffable texi files; I may
follow up with additional series along this line to make the documentation build
reproducibly.

brian m. carlson (7):
  Documentation: fix warning in cat-texi.perl
  Documentation: modernize cat-texi.perl
  Documentation: remove unneeded argument in cat-texi.perl
  Documentation: sort sources for gitman.texi
  Documentation: add XSLT to fix DocBook for Texinfo
  Documentation: move dblatex arguments into variable
  Makefile: add a knob to enable the use of Asciidoctor

 Documentation/Makefile      | 22 ++++++++++++++++++----
 Documentation/cat-texi.perl | 21 ++++++++++++---------
 Documentation/texi.xsl      | 26 ++++++++++++++++++++++++++
 Makefile                    |  6 ++++++
 4 files changed, 62 insertions(+), 13 deletions(-)
 create mode 100644 Documentation/texi.xsl

The newly-added use of the warnings pragma exposes that the $menu[0]
argument to printf has long been silently ignored, since there is no
format specifier for it.  It doesn't appear that the argument is
actually needed, either: there is no reason to insert the name of one
particular documentation page anywhere in the header that's being
generated.

Remove the unused argument, and since the format specification
functionality is no longer needed, convert the printf to a simple print.

Signed-off-by: brian m. carlson <redacted>
---
 Documentation/cat-texi.perl | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/cat-texi.perl b/Documentation/cat-texi.perl
index 1bc84d3c7..14d2f8341 100755
--- a/Documentation/cat-texi.perl
+++ b/Documentation/cat-texi.perl

@@ -20,7 +20,7 @@ while (<STDIN>) {
 }
 close $tmp;
 
-printf '\input texinfo
+print '\input texinfo
 @setfilename gitman.info
 @documentencoding UTF-8
 @dircategory Development

@@ -31,7 +31,7 @@ printf '\input texinfo
 @top Git Manual Pages
 @documentlanguage en
 @menu
-', $menu[0];
+';
 
 for (@menu) {
 	print "* ${_}::\n";

Good style for Perl includes using the strict and warnings pragmas, and
preferring lexical file handles over bareword file handles.  Using
lexical file handles necessitates being explicit when $_ is printed, so
that Perl does not get confused and instead print the glob ref.

The benefit of this modernization is that a formerly obscured bug is now
visible, which will be fixed in a followup patch.

Signed-off-by: brian m. carlson <redacted>
---
 Documentation/cat-texi.perl | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/Documentation/cat-texi.perl b/Documentation/cat-texi.perl
index b1fe52e8b..1bc84d3c7 100755
--- a/Documentation/cat-texi.perl
+++ b/Documentation/cat-texi.perl

@@ -1,9 +1,12 @@
 #!/usr/bin/perl -w
 
+use strict;
+use warnings;
+
 my @menu = ();
 my $output = $ARGV[0];
 
-open TMP, '>', "$output.tmp";
+open my $tmp, '>', "$output.tmp";
 
 while (<STDIN>) {
 	next if (/^\\input texinfo/../\@node Top/);

@@ -13,9 +16,9 @@ while (<STDIN>) {
 	}
 	s/\(\@pxref\{\[(URLS|REMOTES)\]}\)//;
 	s/\@anchor\{[^{}]*\}//g;
-	print TMP;
+	print $tmp $_;
 }
-close TMP;
+close $tmp;
 
 printf '\input texinfo
 @setfilename gitman.info

@@ -34,10 +37,10 @@ for (@menu) {
 	print "* ${_}::\n";
 }
 print "\@end menu\n";
-open TMP, '<', "$output.tmp";
-while (<TMP>) {
+open $tmp, '<', "$output.tmp";
+while (<$tmp>) {
 	print;
 }
-close TMP;
+close $tmp;
 print "\@bye\n";
 unlink "$output.tmp";

Newer versions of Perl produce the warning "Unescaped left brace in
regex is deprecated, passed through in regex" when an unescaped left
brace occurs in a regex.  Escape the brace to avoid this warning.

Signed-off-by: brian m. carlson <redacted>
---
 Documentation/cat-texi.perl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/cat-texi.perl b/Documentation/cat-texi.perl
index 87437f8a9..b1fe52e8b 100755
--- a/Documentation/cat-texi.perl
+++ b/Documentation/cat-texi.perl

@@ -11,7 +11,7 @@ while (<STDIN>) {
 	if (s/^\@top (.*)/\@node $1,,,Top/) {
 		push @menu, $1;
 	}
-	s/\(\@pxref{\[(URLS|REMOTES)\]}\)//;
+	s/\(\@pxref\{\[(URLS|REMOTES)\]}\)//;
 	s/\@anchor\{[^{}]*\}//g;
 	print TMP;
 }

Sorting the sources makes it easier to compare the output using diff.
In addition, it aids groups creating reproducible builds, as the order
of the files is no longer dependent on the file system or other
irrelevant factors.

Signed-off-by: brian m. carlson <redacted>
---
 Documentation/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index a9fb497b8..6e6c82409 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile

@@ -373,7 +373,7 @@ user-manual.pdf: user-manual.xml
 
 gitman.texi: $(MAN_XML) cat-texi.perl
 	$(QUIET_DB2TEXI)$(RM) $@+ $@ && \
-	($(foreach xml,$(MAN_XML),$(DOCBOOK2X_TEXI) --encoding=UTF-8 \
+	($(foreach xml,$(sort $(MAN_XML)),$(DOCBOOK2X_TEXI) --encoding=UTF-8 \
 		--to-stdout $(xml) &&) true) > $@++ && \
 	$(PERL_PATH) cat-texi.perl $@ <$@++ >$@+ && \
 	rm $@++ && \

While Git has traditionally built its documentation using AsciiDoc, some
people wish to use Asciidoctor for speed or other reasons.  Add a
Makefile knob, USE_ASCIIDOCTOR, that sets various options in order to
produce acceptable output.  For HTML output, XHTML5 was chosen, since
the AsciiDoc options also produce XHTML, albeit XHTML 1.1.

Asciidoctor does not have built-in support for the linkgit macro, but it
is available using the Asciidoctor Extensions Lab.  Add a macro to
enable the use of this extension if it is available.  Without it, the
linkgit macros are emitted into the output.

Signed-off-by: brian m. carlson <redacted>
---
 Documentation/Makefile | 12 ++++++++++++
 Makefile               |  6 ++++++
 2 files changed, 18 insertions(+)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 0f4db48eb..5cbecfa99 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile

@@ -174,6 +174,18 @@ ifdef GNU_ROFF
 XMLTO_EXTRA += -m manpage-quote-apos.xsl
 endif
 
+ifdef USE_ASCIIDOCTOR
+ASCIIDOC = asciidoctor
+ASCIIDOC_CONF =
+ASCIIDOC_HTML = xhtml5
+ASCIIDOC_DOCBOOK = docbook45
+ifdef ASCIIDOCTOR_EXTENSIONS_LAB
+ASCIIDOC_EXTRA = -I$(ASCIIDOCTOR_EXTENSIONS_LAB) -rasciidoctor/extensions -rman-inline-macro
+endif
+ASCIIDOC_EXTRA += -alitdd='&\#x2d;&\#x2d;'
+DBLATEX_COMMON =
+endif
+
 SHELL_PATH ?= $(SHELL)
 # Shell quote;
 SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))

diff --git a/Makefile b/Makefile
index 27afd0f37..7ed9d4d4b 100644
--- a/Makefile
+++ b/Makefile

@@ -250,6 +250,12 @@ all::
 # apostrophes to be ASCII so that cut&pasting examples to the shell
 # will work.
 #
+# Define USE_ASCIIDOCTOR to use Asciidoctor instead of AsciiDoc to build the
+# documentation.
+#
+# Define ASCIIDOCTOR_EXTENSIONS_LAB to point to the location of the Asciidoctor
+# Extensions Lab if you have it available.
+#
 # Define PERL_PATH to the path of your Perl binary (usually /usr/bin/perl).
 #
 # Define NO_PERL_MAKEMAKER if you cannot use Makefiles generated by perl's

There are two ways to create a section in a reference document (i.e.,
manpage) in DocBook 4: refsection elements and refsect, refsect2, and
refsect3 elements.  Either form is acceptable as of DocBook 4.2, but
they cannot be mixed.  Prior to DocBook 4.2, only the numbered forms
were acceptable.

docbook2texi only accepts the numbered forms, and this has not generally
been a problem, since AsciiDoc produces the numbered forms.
Asciidoctor, on the other hand, uses a shared backend for DocBook 4 and
5, and uses the unnumbered refsection elements instead.

If we don't convert the unnumbered form to the numbered form,
docbook2texi omits section headings, which is undesirable.  Add an XSLT
stylesheet to transform the unnumbered forms to the numbered forms
automatically, and preprocess the DocBook XML as part of the
transformation to Texinfo format.

Note that this transformation is only necessary for Texinfo, since
docbook2texi provides its own stylesheets.  The DocBook stylesheets,
which we use for other formats, provide the full range of DocBook 4 and
5 compatibility, and don't have this issue.

Signed-off-by: brian m. carlson <redacted>
---
 Documentation/Makefile |  7 ++++---
 Documentation/texi.xsl | 26 ++++++++++++++++++++++++++
 2 files changed, 30 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/texi.xsl

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 6e6c82409..76be7017c 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile

@@ -371,10 +371,11 @@ user-manual.pdf: user-manual.xml
 	$(DBLATEX) -o $@+ -p $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.xsl -s $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.sty $< && \
 	mv $@+ $@
 
-gitman.texi: $(MAN_XML) cat-texi.perl
+gitman.texi: $(MAN_XML) cat-texi.perl texi.xsl
 	$(QUIET_DB2TEXI)$(RM) $@+ $@ && \
-	($(foreach xml,$(sort $(MAN_XML)),$(DOCBOOK2X_TEXI) --encoding=UTF-8 \
-		--to-stdout $(xml) &&) true) > $@++ && \
+	($(foreach xml,$(sort $(MAN_XML)),xsltproc -o $(xml)+ texi.xsl $(xml) && \
+		$(DOCBOOK2X_TEXI) --encoding=UTF-8 --to-stdout $(xml)+ && \
+		rm $(xml)+ &&) true) > $@++ && \
 	$(PERL_PATH) cat-texi.perl $@ <$@++ >$@+ && \
 	rm $@++ && \
 	mv $@+ $@

diff --git a/Documentation/texi.xsl b/Documentation/texi.xsl
new file mode 100644
index 000000000..0f8ff07ec
--- /dev/null
+++ b/Documentation/texi.xsl

@@ -0,0 +1,26 @@
+<!-- texi.xsl:
+     convert refsection elements into refsect elements that docbook2texi can
+     understand -->
+<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+		version="1.0">
+
+<xsl:output method="xml"
+	    encoding="UTF-8"
+	    doctype-public="-//OASIS//DTD DocBook XML V4.5//EN"
+	    doctype-system="http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" />
+
+<xsl:template match="//refsection">
+	<xsl:variable name="element">refsect<xsl:value-of select="count(ancestor-or-self::refsection)" /></xsl:variable>
+	<xsl:element name="{$element}">
+		<xsl:apply-templates select="@*|node()" />
+	</xsl:element>
+</xsl:template>
+
+<!-- Copy all other nodes through. -->
+<xsl:template match="node()|@*">
+	<xsl:copy>
+		<xsl:apply-templates select="@*|node()" />
+	</xsl:copy>
+</xsl:template>
+
+</xsl:stylesheet>

Our dblatex invocation uses several style components from the AsciiDoc
distribution, but those components are not available when building with
Asciidoctor.  Move the command line arguments into a variable so it can
be overridden by the user or makefile configuration options.

Signed-off-by: brian m. carlson <redacted>
---
 Documentation/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 76be7017c..0f4db48eb 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile

@@ -120,6 +120,7 @@ INSTALL_INFO = install-info
 DOCBOOK2X_TEXI = docbook2x-texi
 DBLATEX = dblatex
 ASCIIDOC_DBLATEX_DIR = /etc/asciidoc/dblatex
+DBLATEX_COMMON = -p $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.xsl -s $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.sty
 ifndef PERL_PATH
 	PERL_PATH = /usr/bin/perl
 endif

@@ -368,7 +369,7 @@ user-manual.texi: user-manual.xml
 
 user-manual.pdf: user-manual.xml
 	$(QUIET_DBLATEX)$(RM) $@+ $@ && \
-	$(DBLATEX) -o $@+ -p $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.xsl -s $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.sty $< && \
+	$(DBLATEX) $-o $@+ (DBLATEX_COMMON) $< && \
 	mv $@+ $@
 
 gitman.texi: $(MAN_XML) cat-texi.perl texi.xsl

On 2017-01-21 21:59:11, brian m. carlson wrote:

Our dblatex invocation uses several style components from the AsciiDoc
distribution, but those components are not available when building with
Asciidoctor.  Move the command line arguments into a variable so it can
be overridden by the user or makefile configuration options.
[...]
-	$(DBLATEX) -o $@+ -p $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.xsl -s $(ASCIIDOC_DBLATEX_DIR)/asciidoc-dblatex.sty $< && \
+	$(DBLATEX) $-o $@+ (DBLATEX_COMMON) $< && \

It looks as the dollar sign is in the wrong place, should be in front of 
(DBLATEX_COMMON).

Regards,
Øyvind

+-| Øyvind A. Holm [off-list ref] - N 60.37604° E 5.33339° |-+
| OpenPGP: 0xFB0CBEE894A506E5 - http://www.sunbase.org/pubkey.asc |
| Fingerprint: A006 05D6 E676 B319 55E2  E77E FB0C BEE8 94A5 06E5 |
+------------| 61815930-e03e-11e6-b4f4-db5caa6d21d3 |-------------+