Opened 5 years ago

Closed 5 years ago

#4329 closed defect (wontfix)

PostgreSQL 12 many regression failures rounding and extra warning output

Reported by: robe Owned by: robe
Priority: medium Milestone: PostGIS 2.5.2
Component: postgis Version: master
Keywords: postgres 12 Cc:

Description (last modified by robe)

I'm not sure how long this has been going on. I suspect somewhere between Feb 11th and Feb 15th is when it started.

Debbie has this with pg12 head I've disabled her 12 testing on PostGIS 2.5, but kept it on for 3.0

All look like rounding except for the estimatedextent. I can't tell if it's in 12 whatever we have is running 4 times instead of once or just some artifact of the WARNING system in 12 that it outputs the warning 4 times.

Checking for shp2pgsql ... found
Checking for pgsql2shp ... found
TMPDIR is /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64
Creating database 'postgis_reg' 
Loading PostGIS into 'postgis_reg' 
  /var/lib/jenkins/workspace/postgis/branches/3.0/regress/00-regress-install/share/contrib/postgis/postgis.sql
  /var/lib/jenkins/workspace/postgis/branches/3.0/regress/00-regress-install/share/contrib/postgis/postgis_comments.sql
  /var/lib/jenkins/workspace/postgis/branches/3.0/regress/00-regress-install/share/contrib/postgis/postgis_proc_set_search_path.sql
PostgreSQL 12devel on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.2.0-9) 8.2.0, 64-bit
  Postgis 3.0.0dev - r17261 - 2019-02-19 04:35:15
  scripts 3.0.0dev r17261
  GEOS: 3.7.2dev-CAPI-1.11.2 0cd06b20
  PROJ: Rel. 5.2.0, September 15th, 2018

Running tests

 ../loader/Point .............. ok 
 ../loader/PointM .............. ok 
 ../loader/PointZ .............. ok 
 ../loader/MultiPoint .............. ok 
 ../loader/MultiPointM .............. ok 
 ../loader/MultiPointZ .............. ok 
 ../loader/Arc .............. ok 
 ../loader/ArcM .............. ok 
 ../loader/ArcZ .............. ok 
 ../loader/Polygon .............. ok 
 ../loader/PolygonM .............. ok 
 ../loader/PolygonZ .............. ok 
 ../loader/TSTPolygon ......... ok 
 ../loader/TSIPolygon ......... ok 
 ../loader/TSTIPolygon ......... ok 
 ../loader/PointWithSchema ..... ok 
 ../loader/NoTransPoint ......... ok 
 ../loader/NotReallyMultiPoint ......... ok 
 ../loader/MultiToSinglePoint ......... ok 
 ../loader/ReprojectPts ........ ok 
 ../loader/ReprojectPtsGeog ........ ok 
 ../loader/Latin1 .... ok 
 ../loader/Latin1-implicit .... ok 
 ../loader/mfile .... ok 
 ../dumper/literalsrid ....... ok 
 ../dumper/realtable ....... ok 
 affine .. ok 
 bestsrid .. ok 
 binary .. ok 
 boundary .. ok 
 chaikin .. ok 
 filterm .. ok 
 cluster .. ok 
 concave_hull .. ok 
 concave_hull_hard .. ok 
 ctors .. ok 
 curvetoline .. ok 
 dump .. ok 
 dumppoints .. ok 
 empty .. ok 
 estimatedextent .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_41_diff)
-----------------------------------------------------------------------------
--- estimatedextent_expected	2019-01-23 17:21:20.650097294 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_41_out	2019-02-19 04:38:35.021379610 +0000
@@ -6,17 +6,32 @@
 WARNING:  stats for "t.g" do not exist
 #877.2.deprecated|
 WARNING:  stats for "t.g" do not exist
+WARNING:  stats for "t.g" do not exist
+WARNING:  stats for "t.g" do not exist
+WARNING:  stats for "t.g" do not exist
 #877.3||||
 #877.4|-10.15000|20.15000|-50.40000|30.40000
 #877.5|-10.15000|20.15000|-50.40000|30.40000
 WARNING:  stats for "c1.g" do not exist
+WARNING:  stats for "c1.g" do not exist
+WARNING:  stats for "c1.g" do not exist
+WARNING:  stats for "c1.g" do not exist
 #3391.1||||
 WARNING:  stats for "c2.g" do not exist
+WARNING:  stats for "c2.g" do not exist
+WARNING:  stats for "c2.g" do not exist
+WARNING:  stats for "c2.g" do not exist
 #3391.2||||
 WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
 #3391.3||||
 #3391.4|0.00|1.00|0.00|1.00
 WARNING:  stats for "c2.g" do not exist
+WARNING:  stats for "c2.g" do not exist
+WARNING:  stats for "c2.g" do not exist
+WARNING:  stats for "c2.g" do not exist
 #3391.5||||
 #3391.6|0.00|1.00|0.00|1.00
 #3391.7|0.00|1.00|0.00|1.00
@@ -26,15 +41,30 @@
 #3391.11|-1.00|0.00|-1.00|0.00
 #3391.12|-1.01|2.02|-1.01|2.02
 WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
 #3391.13||||
 WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
 #3391.14||||
 WARNING:  stats for "c1.g" do not exist
+WARNING:  stats for "c1.g" do not exist
+WARNING:  stats for "c1.g" do not exist
+WARNING:  stats for "c1.g" do not exist
 #3391.15||||
 WARNING:  stats for "c1.g" do not exist
+WARNING:  stats for "c1.g" do not exist
+WARNING:  stats for "c1.g" do not exist
+WARNING:  stats for "c1.g" do not exist
 #3391.16||||
 #3391.17|0.00|1.00|0.00|1.00
 WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
+WARNING:  stats for "p.g" do not exist
 #3391.18||||
 #3391.19|0.00|1.00|0.00|1.00
 #3391.20|0.00|1.00|0.00|1.00
-----------------------------------------------------------------------------
 forcecurve .. ok 
 geography .. ok 
 geometric_median .. ok 
 hausdorff .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_45_diff)
-----------------------------------------------------------------------------
--- hausdorff_expected	2018-10-19 23:06:41.113379730 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_45_out	2019-02-19 04:38:35.293380987 +0000
@@ -1,6 +1,6 @@
-hausdorff_poly_poly|0.707106781186548
+hausdorff_poly_poly|0.7071067811865476
 hausdorff_ls_ls|1
 hausdorff_ls_ls_2|2
 hausdorff_ls_mp|1
-hausdorff_ls_ls_3|14.142135623731
+hausdorff_ls_ls_3|14.142135623730951
 hausdorffdensify_ls_ls|70
-----------------------------------------------------------------------------
 in_geohash .. ok 
 in_gml .. ok 
 in_kml .. ok 
 in_encodedpolyline .. ok 
 iscollection .. ok 
 knn_recheck .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_51_diff)
-----------------------------------------------------------------------------
--- knn_recheck_expected	2018-10-19 23:06:41.065380052 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_51_out	2019-02-19 04:38:57.481493351 +0000
@@ -103,5 +103,5 @@
 #3nd-3|600001|9749|54.5453|54.5453
 #3nd-3|600001|10041|54.6233|54.6233
 #3573|8
-#3418|0.331823813642119|0.331823813642119
+#3418|0.33182381364211927|0.33182381364211927
 #3418|0.55|0.55
-----------------------------------------------------------------------------
 legacy .. ok 
 long_xact .. ok 
 lwgeom_regress .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_54_diff)
-----------------------------------------------------------------------------
--- lwgeom_regress_expected	2018-10-19 23:06:41.121379675 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_54_out	2019-02-19 04:38:58.309497543 +0000
@@ -28,8 +28,8 @@
 ERROR:  Argument must be POINT geometries
 ERROR:  parse error - invalid geometry
 ST_Angle_4_pts|4.71238898038469
-ST_Angle_4_pts|0.785398163397448
-ST_Angle_3_pts|1.5707963267949
+ST_Angle_4_pts|0.7853981633974483
+ST_Angle_3_pts|1.5707963267948966
 ERROR:  Operation on mixed SRID geometries
 ERROR:  Empty geometry
 ST_Angle_2_lines|4.71238898038469
-----------------------------------------------------------------------------
 measures .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_55_diff)
-----------------------------------------------------------------------------
--- measures_expected	2019-02-18 08:55:22.999999325 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_55_out	2019-02-19 04:38:58.381497909 +0000
@@ -1,9 +1,9 @@
 113|291
 114|140
 115|140
-116|4.24264068711929
-117|4.24264068711929
-118|5.19615242270663
+116|4.242640687119286
+117|4.242640687119286
+118|5.196152422706632
 134|0
 135|13
 136|13
@@ -16,7 +16,7 @@
 st_maxdistance_134|0
 st_maxdistance_135|13
 st_maxdistance_136|13
-st_maxdistance_dist|22.3606797749979|22.3606797749979
+st_maxdistance_dist|22.360679774997898|22.360679774997898
 st_longestline_134|LINESTRING(1 2,1 2)
 st_longestline_135|LINESTRING(5 0,10 12)
 st_longestline_136|LINESTRING(0 0,5 12)
@@ -25,14 +25,14 @@
 distancetest2|0|50|0.0000000000|0.0000000000|0.0000000000|0.0000000000|LINESTRING(-40 -20,-10 20)|LINESTRING(-10 20,-40 -20)
 distancepoly1|1|50|LINESTRING(17 18,17 19)|LINESTRING(17 19,17 18)|LINESTRING(29 39,-1 -1)|LINESTRING(-1 -1,29 39)
 distancepoly2|0|26.1725046566048|LINESTRING(17 14,17 14)|LINESTRING(17 14,17 14)|LINESTRING(17 18,-1 -1)|LINESTRING(-1 -1,17 18)
-distancepoly3|0|26.9072480941474|LINESTRING(17 19,17 19)|LINESTRING(17 19,17 19)|LINESTRING(17 19,-1 -1)|LINESTRING(-1 -1,17 19)
-distancepoly4|0|28.3196045170126|LINESTRING(16 19,16 19)|LINESTRING(16 19,16 19)|LINESTRING(18 20,-1 -1)|LINESTRING(-1 -1,18 20)
+distancepoly3|0|26.90724809414742|LINESTRING(17 19,17 19)|LINESTRING(17 19,17 19)|LINESTRING(17 19,-1 -1)|LINESTRING(-1 -1,17 19)
+distancepoly4|0|28.319604517012593|LINESTRING(16 19,16 19)|LINESTRING(16 19,16 19)|LINESTRING(18 20,-1 -1)|LINESTRING(-1 -1,18 20)
 distancepoly5|0|26.1725046566048|LINESTRING(17 12,17 12)|LINESTRING(17 12,17 12)|LINESTRING(17 18,-1 -1)|LINESTRING(-1 -1,17 18)
-distancepoly6|0|32.5269119345812|LINESTRING(2 2,2 2)|LINESTRING(2 2,2 2)|LINESTRING(2 2,25 25)|LINESTRING(25 25,2 2)
-3dDistancetest1|6.40312423743285|6.40312423743285|f|f|LINESTRING(1 1 1,3 2 7)|POINT(1 1 1)|LINESTRING(1 1 1,3 2 7)
-3dDistancetest2|0|1.73205080756888|t|t|LINESTRING(1 1 1,1 1 1)|POINT(1 1 1)|LINESTRING(1 1 1,0 0 0)
+distancepoly6|0|32.526911934581186|LINESTRING(2 2,2 2)|LINESTRING(2 2,2 2)|LINESTRING(2 2,25 25)|LINESTRING(25 25,2 2)
+3dDistancetest1|6.4031242374328485|6.4031242374328485|f|f|LINESTRING(1 1 1,3 2 7)|POINT(1 1 1)|LINESTRING(1 1 1,3 2 7)
+3dDistancetest2|0|1.7320508075688772|t|t|LINESTRING(1 1 1,1 1 1)|POINT(1 1 1)|LINESTRING(1 1 1,0 0 0)
 3dDistancetest3|4.09994192757944|6.48074069840786|t|f|LINESTRING(1 1 1,0.61904761904762 -0.19047619047619 4.90476190476191)|POINT(1 1 1)|LINESTRING(1 1 1,5 2 6)
-3dDistancetest4|2|10.0498756211209|t|f|LINESTRING(1 1 3,1 1 1)|POINT(1 1 3)|LINESTRING(5 7 8,1 1 1)
+3dDistancetest4|2|10.04987562112089|t|f|LINESTRING(1 1 3,1 1 1)|POINT(1 1 3)|LINESTRING(5 7 8,1 1 1)
 3dDistancetest5|2|10|t|f|LINESTRING(5 0 5,5 2 5)|POINT(5 0 5)|LINESTRING(11 0 5,5 0 13)
 3dDistancetest6|0
 3dDistancetest7|0
-----------------------------------------------------------------------------
 minimum_bounding_circle .. ok 
 normalize .. ok 
 operators .. ok 
 orientation .. ok 
 out_geometry .. ok 
 out_geography .. ok 
 polygonize .. ok 
 polyhedralsurface .. ok 
 postgis_type_name .. ok 
 quantize_coordinates .. ok 
 regress .. ok 
 regress_bdpoly .. ok 
 regress_buffer_params .. ok 
 regress_gist_index_nd .. ok 
 regress_index .. ok 
 regress_index_nulls .. ok 
 regress_management .. ok 
 regress_selectivity .. ok 
 regress_lrs .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_74_diff)
-----------------------------------------------------------------------------
--- regress_lrs_expected	2018-12-03 21:53:22.510210503 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_74_out	2019-02-19 04:39:03.129521953 +0000
@@ -20,7 +20,7 @@
 #4155.1|MULTILINESTRING Z ((2 2 2,5 5 5))
 #4155.2|TIN Z EMPTY
 #4155.3|MULTIPOLYGON Z EMPTY
-line_locate_point_1|0.528602749909894
+line_locate_point_1|0.5286027499098938
 line_locate_point_2|1
 line_locate_point_3|0
 line_locate_point_4|0
-----------------------------------------------------------------------------
 regress_ogc .. ok 
 regress_ogc_cover .. ok 
 regress_ogc_prep .. ok 
 regress_proj .. ok 
 relate .. ok 
 remove_repeated_points .. ok 
 removepoint .. ok 
 reverse .. ok 
 setpoint .. ok 
 simplify .. ok 
 simplifyvw .. ok 
 size .. ok 
 snaptogrid .. ok 
 split .. ok 
 sql-mm-serialize .. ok 
 sql-mm-circularstring .. ok 
 sql-mm-compoundcurve .. ok 
 sql-mm-curvepoly .. ok 
 sql-mm-general .. ok 
 sql-mm-multicurve .. ok 
 sql-mm-multisurface .. ok 
 swapordinates .. ok 
 summary .. ok 
 temporal .. ok 
 temporal_knn .. ok 
 tickets .. ok 
 twkb .. ok 
 typmod .. ok 
 wkb .. ok 
 wkt .. ok 
 wmsservers .. ok 
 offsetcurve .. ok 
 relatematch .. ok 
 isvaliddetail .. ok 
 sharedpaths .. ok 
 snap .. ok 
 node .. ok 
 unaryunion .. ok 
 clean .. ok 
 relate_bnr .. ok 
 delaunaytriangles .. ok 
 clipbybox2d .. ok 
 subdivide .. ok 
 voronoi .. ok 
 regress_brin_index .. ok 
 regress_brin_index_3d .. ok 
 regress_brin_index_geography .. ok 
 minimum_clearance .. ok 
 oriented_envelope .. ok 
 frechet .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_124_diff)
-----------------------------------------------------------------------------
--- frechet_expected	2018-10-19 23:06:40.889381235 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/3_0_pg12w64/test_124_out	2019-02-19 04:39:27.985647831 +0000
@@ -1,5 +1,5 @@
 frechet_ls_ls|1
 frechet_ls_ls_2|2.23606797749979
 frechet_ls_mp|1
-frechet_ls_ls_3|70.7106781186548
+frechet_ls_ls_3|70.71067811865476
 frechetdensify_ls_ls|50
-----------------------------------------------------------------------------
 in_geojson .. ok 
 regress_spgist_index_2d .. ok 
 regress_spgist_index_3d .. ok 
 regress_spgist_index_nd .. ok 
 mvt .. ok 
 geobuf .. ok 
 mvt_jsonb .. ok 
 uninstall .  /var/lib/jenkins/workspace/postgis/branches/3.0/regress/00-regress-install/share/contrib/postgis/uninstall_postgis.sql
. ok (4679)

Run tests: 132
Failed: 7
make[2]: *** [Makefile:212: check] Error 7
make[2]: Leaving directory '/var/lib/jenkins/workspace/postgis/branches/3.0/regress/core'
make[1]: *** [Makefile:28: check] Error 2
make[1]: Leaving directory '/var/lib/jenkins/workspace/postgis/branches/3.0/regress'
make: *** [GNUmakefile:20: check] Error 1

I compiled PostgreSQL 12 and get same failures.

at a glance looks like mostly rounding issues that perhaps buttoning up the tests with some grace for floats will solve.

Change History (15)

comment:1 by robe, 5 years ago

Description: modified (diff)
Summary: PostgreSQL 12 many regression failures Look like roundingPostgreSQL 12 many regression failures rounding and extra warning output

comment:2 by robe, 5 years ago

Owner: changed from pramsey to robe

comment:3 by robe, 5 years ago

In 17262:

Put in rounding to fix regression as result of changes in PostgreSQL 12
References #4329 for PostGIS 3.0

comment:4 by strk, 5 years ago

How does PostgreSQL affect rounding ? Is that because we're inheriting compiler flags ?

comment:5 by komzpa, 5 years ago

PostgreSQL switched to Ryu library to format number output. Maybe we need to consider it too?

comment:6 by robe, 5 years ago

Owner: changed from robe to pramsey

I put in rounding to compensate for all except estimatedextents - passing that off to pramsey

comment:7 by robe, 5 years ago

Owner: changed from pramsey to robe

Taking it back. I suspect the estimatedextent extra WARNINGS is because of change in CTE. That CTEs are inlined now. So there is probably an extra warning output for each column.

comment:8 by robe, 5 years ago

In 17267:

Change from using CTE to use subquery so results are the same across all platforms
reason necessary in PostgreSQL 12, CTE changed from being always materialized to being often inlined.
This means PG12, no longer behaves like old CTE and behaves more like subquery.
As a result 12 CTE is outputting 4 warnings instead of 1 like older versions.
Changing to just subquery should force all to output 4 warnings.
references #4329 for PostGIS 3.0

comment:9 by robe, 5 years ago

In 17268:

Missed some spots in last commit
Change from using CTE to use subquery so results are the same across all platforms
reason necessary in PostgreSQL 12, CTE changed from being always materialized to being often inlined.
This means PG12, no longer behaves like old CTE and behaves more like subquery.
As a result 12 CTE is outputting 4 warnings instead of 1 like older versions.
Changing to just subquery should force all to output 4 warnings.
references #4329 for PostGIS 3.0

comment:10 by pramsey, 5 years ago

We could "just" put "set extra_float_digits=0" at the top of the offending regression files, or at the top of run_test itself… apparently that returns float8 output to the original number of digits.

comment:11 by pramsey, 5 years ago

I have commited a change to run_test.pl that adds this option in both trunk r17269 and 2.5 r17270. We may or may not want to back out all the rounding in trunk. There are still failures in trunk raster regression due, I'm pretty sure, to the change in CTE behaviour.

comment:12 by robe, 5 years ago

Let's not bother backing out the old changes. For the ones we have I think we've also in the past suffered from platform differences. So checking out to 12 digits is not really needed and if our algs change that also tends to schew those extra digits that are meaningless. And I'm just too damn lazy to backout.

comment:13 by robe, 5 years ago

Next to fix is the raster CTEs. I'm holding off on putting in a fix such as offset 0 or switching to using LATERAL for these to see if they make any changes to my over-dramatized complaint

https://www.postgresql.org/message-id/flat/000001d4caed%24d29b9ae0%2477d2d0a0%24%40pcorp.us

If they fix the functional expansion thing to not inline, then the tests should just start passing on their own. I haven't looked to see what dirt sits under the topology covers, but I expect there to be some.

comment:14 by robe, 5 years ago

In 17279:

For CTEs to materialize by putting in OFFSET 0 on raster CTE queries
So PostgreSQL CTE call behaves like older versions that always materialized
references #4329 for PostGIS 3.0

comment:15 by robe, 5 years ago

Resolution: wontfix
Status: newclosed

I here by say - I really don't want PostGIS 2.5 working with PostgreSQL 12 — REALLY REALLY

Note: See TracTickets for help on using tickets.