PostGIS 2.4.4 rt_gdalwrap test fails with GDAL 2.3.0 & PROJ 5.1.0 on Debian unstable

[sebastic]

As reported by Niko Tyni in ​Debian Bug #901158:

This package fails to build from source on current sid/amd64.

 ### /tmp/pgis_reg/test_50_diff ###
--- rt_gdalwarp_expected      2018-04-06 05:05:52.000000000 +0000
+++ /tmp/pgis_reg/test_50_out 2018-06-09 14:27:15.049036062 +0000
@@ -19,8 +19,8 @@
 0.17|993309|243|243|1|50.000|50.000|0.000|0.000|950760.000|1396957.000|t|t|t
 0.18|992163|10|10|1|1000.000|-1000.000|3.000|3.000|-500030.000|600000.000|t|t|t
 0.19|993310|12|12|1|1009.894|-1009.894|3.000|3.000|950691.792|1409281.783|t|t|t
-0.2|993309|12|12|1|1009.916|-1009.916|0.000|0.000|950762.305|1409088.896|t|t|t
-0.20|993309|12|12|1|1009.916|-1009.916|1.000|3.000|950742.107|1409088.896|t|t|t
+0.2|993309|12|12|1|1009.876|-1009.876|0.000|0.000|950808.447|1409091.640|t|t|t
+0.20|993309|12|12|1|1009.876|-1009.876|1.000|3.000|950788.250|1409091.640|t|t|t
 0.21|993310|24|24|1|500.000|500.000|3.000|3.000|950657.188|1397356.783|t|t|t
 0.22|993310|26|26|1|500.000|500.000|0.000|6.000|950452.000|1396632.000|t|t|t
 0.23|984269|12|8|1|0.012|-0.012|0.000|0.000|-107.029|50.206|t|t|t
@@ -63,12 +63,12 @@
 1.9|992163|11|10|1|1000.000|-1000.000|0.000|0.000|-500001.000|600000.000|t|t|t
 2.1|993310|12|12|1|1009.894|-1009.894|0.000|0.000|950732.188|1409281.783|t|t|t
 2.10|993310|24|24|1|500.000|500.000|0.000|0.000|950732.188|1397281.783|t|t|t
-2.11|993309|121|121|1|100.000|100.000|0.000|0.000|950762.305|1396988.896|t|t|t
+2.11|993309|121|121|1|100.000|100.000|0.000|0.000|950808.447|1396991.640|t|t|t
 2.12|993310|6|6|1|2000.000|2000.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.13|993310|8|8|1|1500.000|1500.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.14|993310|24|24|1|500.000|500.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.15|993310|16|16|1|750.000|750.000|0.000|0.000|950732.188|1397281.783|t|t|t
-2.2|993309|12|12|1|1009.916|-1009.916|0.000|0.000|950762.305|1409088.896|t|t|t
+2.2|993309|12|12|1|1009.876|-1009.876|0.000|0.000|950808.447|1409091.640|t|t|t
 2.3|994269|12|8|1|0.012|-0.012|0.000|0.000|-107.029|50.206|t|t|t
 2.4|||||||||||||
 2.5|993310|12|12|1|1009.894|-1009.894|0.000|0.000|950732.188|1409281.783|t|t|t
 ### end of log dumps ###
make[1]: *** [debian/rules:200: override_dh_auto_test] Error 2
make[1]: Leaving directory '/<<BUILDDIR>>/postgis-2.4.4+dfsg'
make: *** [debian/rules:111: build] Error 2

Full build log attached.

​postgis_amd64-2018-06-09T14:19:48Z.build.gz

[sebastic]

The attached ​rt_gdalwarp.patch fixes the issue, but is likely only appropriate for systems with GDAL 2.3.0 and PROJ 5.1.0.

The underlaying issue may be related to ​pj_transform: reset error state before each call to pj_inv/pj_fwd for which Even mentioned its affect on some gdalwarp scenarios.

[sebastic]

This patch is still applied to postgis-2.5beta1

[robe]

In 16682:

Change srid test for 993309 to be the same as 3309 in spatial_ref_sys table.
References #4104

[robe]

Bas,

I'm not setup to test GDAL 2.3.0 or Proj 5.1.0 at the moment. I noticed though that the test 993309 isn't quite what we are shipping with spatial_ref_sys. Though making it the same didn't change the answer I get.

Can you see if that helps your regress any?

If we can't come up with a workable test for both versions of proj, I suppose we can just take out that particular srid test.

[sebastic]

Disabling rt_gdalwarp.patch and applying the changes from r16682 makes no difference:

 rt_gdalwarp .. failed (diff expected obtained: /tmp/pgis_reg/test_52_diff)
-----------------------------------------------------------------------------
--- rt_gdalwarp_expected        2018-07-03 20:26:25.000000000 +0000
+++ /tmp/pgis_reg/test_52_out   2018-08-07 11:01:22.106541719 +0000
@@ -19,8 +19,8 @@
 0.17|993309|243|243|1|50.000|50.000|0.000|0.000|950760.000|1396957.000|t|t|t
 0.18|992163|10|10|1|1000.000|-1000.000|3.000|3.000|-500030.000|600000.000|t|t|t
 0.19|993310|12|12|1|1009.894|-1009.894|3.000|3.000|950691.792|1409281.783|t|t|t
-0.2|993309|12|12|1|1009.916|-1009.916|0.000|0.000|950762.305|1409088.896|t|t|t
-0.20|993309|12|12|1|1009.916|-1009.916|1.000|3.000|950742.107|1409088.896|t|t|t
+0.2|993309|12|12|1|1009.876|-1009.876|0.000|0.000|950808.447|1409091.640|t|t|t
+0.20|993309|12|12|1|1009.876|-1009.876|1.000|3.000|950788.250|1409091.640|t|t|t
 0.21|993310|24|24|1|500.000|500.000|3.000|3.000|950657.188|1397356.783|t|t|t
 0.22|993310|26|26|1|500.000|500.000|0.000|6.000|950452.000|1396632.000|t|t|t
 0.23|984269|12|8|1|0.012|-0.012|0.000|0.000|-107.029|50.206|t|t|t
@@ -63,12 +63,12 @@
 1.9|992163|11|10|1|1000.000|-1000.000|0.000|0.000|-500001.000|600000.000|t|t|t
 2.1|993310|12|12|1|1009.894|-1009.894|0.000|0.000|950732.188|1409281.783|t|t|t
 2.10|993310|24|24|1|500.000|500.000|0.000|0.000|950732.188|1397281.783|t|t|t
-2.11|993309|121|121|1|100.000|100.000|0.000|0.000|950762.305|1396988.896|t|t|t
+2.11|993309|121|121|1|100.000|100.000|0.000|0.000|950808.447|1396991.640|t|t|t
 2.12|993310|6|6|1|2000.000|2000.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.13|993310|8|8|1|1500.000|1500.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.14|993310|24|24|1|500.000|500.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.15|993310|16|16|1|750.000|750.000|0.000|0.000|950732.188|1397281.783|t|t|t
-2.2|993309|12|12|1|1009.916|-1009.916|0.000|0.000|950762.305|1409088.896|t|t|t
+2.2|993309|12|12|1|1009.876|-1009.876|0.000|0.000|950808.447|1409091.640|t|t|t
 2.3|994269|12|8|1|0.012|-0.012|0.000|0.000|-107.029|50.206|t|t|t
 2.4|||||||||||||
 2.5|993310|12|12|1|1009.894|-1009.894|0.000|0.000|950732.188|1409281.783|t|t|t
-----------------------------------------------------------------------------

This was tested with postgis 2.5.0beta1.

[robe]

Hmm reading thru the thread. Sounds like a bug in proj 5? I assume your PROJ 5.1.0. doesn't have Even's patch in place yet right? In which case when it does this would be giving the answer we have.

I'm going to disable the 993309 srid test for now with a note to turn it back on later. I'll keep this ticket open for the time being though.

[sebastic]

The changes Even mentioned where included as a patch in the proj (5.0.1-2) Debian package, the 5.1.0-rc1 upstream release was the first release to include the changes in proj itself.

The recent development of PROJ has changed coordinate precision quite a bit, many test cases in other projects needed to be adapted to not expect the exact same output as with PROJ 4.x. This seems to be another case like that.

Kristian Evers <kreve@…> should be able to give an authoritative answer about what changed in PROJ 5.x to cause this issue.

[robe]

In 16688:

disable tests that give different answer on Proj.5 vs. Proj.4
References #4104 for PostGIS 2.5.0

[myon]

Is this issue still relevant?

[sebastic]

Re-enabling the test will most likely cause the test target to fail again.

[Algunenano]

Re-enabling the test works in my machine with GDAL 3 and PROJ 6.

I've created a PR to test this in travis (​https://github.com/postgis/postgis/pull/450):

• gdal 2.2 - proj 4.8
• gdal 2.2 - proj 4.9
• gdal 2.3 - proj 4.9
• gdal 2.4 - proj 5.2

[algunenano]

In 17616:

Re-enable gdal tests

Reverts r16688

Closes #4104
Closes ​https://github.com/postgis/postgis/pull/450

[Algunenano]

Gitlab build broke showing thw same issue. It uses PROJ 5.2

[Algunenano]

I've tried changing the definition for 993309 in several ways without any luck.

My local installation all CIs except for Gitlab (which uses Debian buster packages) work fine, and they cover multiple releases of both gdal and proj, so I don't know where this error might be coming from.

I'm going to disable again these tests and leave it be for now.

[algunenano]

In 17619:

Disable tests that give different answer on Proj.5 vs. Proj.4

References #4104