docbook validation preventing regress when trying to access down docbook site

[robe]

I'm having trouble getting winnie or my desktop to regress. Issue as I reckon is that the docbook site is down and the validation check of postgis-out.xml is preventing things from being generated like comments needed for extension build.

Can we turn it off or at least make it conditional. I shouldn't have to be slowed down by things waiting to validate unless I am building the documentation.

here is the error:

make[1]: Entering directory `/projects/postgis/branches/2.2/doc'
/projects/xsltproc/xmllint --loaddtd --xinclude --valid postgis-out.xml > /dev/null
postgis-out.xml:117: warning: failed to load external entity "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
]>
  ^
postgis-out.xml:119: element title: validity error : No declaration for element title
  <title>PostGIS &last_release_version; Manual</title>
                                                      ^
postgis-out.xml:123: element corpauthor: validity error : No declaration for element corpauthor
	<corpauthor>The PostGIS Development Group</corpauthor>
	                                                      ^
postgis-out.xml:125: element firstname: validity error : No declaration for element firstname
		<firstname>Paul</firstname>

[robe]

notice raster_comments is missing —novalid clause so added at r12699 to trunk. Still have outstanding issue of xmllint

[strk]

On debian based systems there's a way to map urls to local files. Here I have an /etc/xml-docbook-xml.xml file with this node:

/etc/xml/docbook-xml.xml:<delegateSystem systemIdStartString="http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" catalog="file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml"/>

The local file was found in the "docbook-xml" package.

That said, I agree the docs should not be attempted to be checked when they are not built, but the checks checks postgis-out.xml which is also used to build comments. This is just to say that it is unlikely that you really don't use postgis-out.xml.

[robe]

Milestone renamed

[Algunenano]

This is hitting again and mixed with the fact that Travis is super slow down, it's a huge pain in the ass.

Docbook, at least mine, seems happy if I download the directory and change the reference to something like this:

diff --git a/doc/postgis.xml b/doc/postgis.xml
index 56087684f..f6577a496 100644
--- a/doc/postgis.xml
+++ b/doc/postgis.xml
@@ -1,6 +1,6 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
-                    "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+                    "./docbook-xml-4.5/docbookx.dtd" [
 
 <!-- This value is automatically generated by the Makefile -->
 <!ENTITY last_release_version "@@LAST_RELEASE_VERSION@@">

So I'm going to try that and if Travis is happy I'll remove the remote access and push the needed files to the repo.

[strk]

I think there's some support for looking for dockbook files on the filesystem, on known locations.

[strk]

On my system, the docbook-xml package installs /usr/share/xml/docbook/schema/dtd/4.5/docbookx.dtd which I belive is used by postgis build scripts without this change

[Algunenano]

That's true.

It seems like something like this would remove the need to download anything since it's already available:

diff --git a/doc/postgis.xml b/doc/postgis.xml
index 782d93b20..8e4ab4908 100644
--- a/doc/postgis.xml
+++ b/doc/postgis.xml
@@ -1,6 +1,6 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
-                    "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+                    "n:/share/xml/docbook/xml-dtd-4.5/docbookx.dtd" [
 
 <!-- This value is automatically generated by the Makefile -->
 <!ENTITY last_release_version "@@LAST_RELEASE_VERSION@@">

[strk]

I'm not sure you need to change the xml file. At least there was not as of [6af019ac523d616a29043d46000ca3d08c93d116/git] — checkout the XSLBASE variable in configure.ac and how it is used in doc/Makefile.in

See also [dd79f4db28bb407dfe13eab94285fbe9ffe71bff/git] as another commit which was addressing finding the DTD on the sytem (with no change to the XML file)

[Algunenano]

It seems part of the problem with Travis is that xmllint is unable to follow a redirect for whatever reason, and the url is redirecting to its https version.

About adding more configure options: I don't really like it as they are not only extremely situations but also finicky. For example, MATHML2_DTD is guessing where some files might be and it's certainly incomplete. Not that it matters since it's not used anywhere.

[Algunenano]

Changing to https didn't work because apparently xmllint does not support https, and that's why it's failing.

In my system, what xmllint does is it tries to open https://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd as a file, which obviously fails. Then it explores the local environment: /etc/xml/catalog → /etc/xml/docbook-xml which leads to /usr/share/xml/docbook/xml-dtd-4.5/docbookx.dtd which works.

So, AFAICS right now, I think that the only way of being able to pass this is to have docbook-xml installed in your local system as the remote url will always fail, but I'm not sure when this started happening. It was fine a few days ago but not yesterday, so it's likely that www.oasis-open.org decided to start redirecting http → https.

There are several solutions to this problem:

• Install docbook-xml in the docker images, which is what I'm planning to do for now.
• Vendor in the necessary files.
• Remove docbook and use something else.

[strk]

Install docbook-xml in the docker images seems the best option to me too.

[Algunenano]

I've built and uploaded the CI docker images for Travis. I still think we should remove network dependencies but I'm not going to work on that right now.

[strk]

I noticed we're using —nonet switch of xsltproc now on make html, but not on other rules, using it always would close this ticket ?

[Sandro Santilli <strk@…>]

In 94922de5/git:

Refuse hitting network for downloading dtd

Closes #2792

[Sandro Santilli <strk@…>]

In eb027f6/git:

Skip hitting network for cheatsheets and comments generation too

References #2792