Opened 7 years ago

Closed 7 years ago

#3759 closed defect (fixed)

PostgreSQL 10 failure on geometry

Reported by: robe Owned by: pramsey
Priority: blocker Milestone: PostGIS 2.4.0
Component: postgis Version: master
Keywords: postgresql 10 Cc:

Description (last modified by robe)

Changing this ticket to refer to geometry 

/tmp/2.4.0dev_pg10_geos3.7.0dev_gdal2.1.3w64/test_89_diff)
-----------------------------------------------------------------------------
--- tickets_expected	2016-09-26 01:59:38 -0400
+++ /projects/postgis/tmp/2.4.0dev_pg10_geos3.7.0dev_gdal2.1.3w64/test_89_out	2017-05-25 15:15:09 -0400
@@ -285,7 +285,7 @@
 #3375|GEOMETRYCOLLECTION(POINT(0 -7))
 #3399|1|1
 #3399|10|10
-#3399|100|100
+#3399|100|99
 #3399|1000|1000
 ERROR:  invalid KML representation
 #3437a|5
-----------------------------------------------------------------------------

The raster problems are largely fixed by #3760. The rt_union one I'll open a separate ticket for. 

Upgrading core
  /projects/postgis/branches/2.4/regress/00-regress-install/share/contrib/postgis/postgis_upgrade.sql
PostgreSQL 10beta1 on x86_64-w64-mingw32, compiled by x86_64-w64-mingw32-gcc.exe (x86_64-win32-seh-rev1, Built by MinGW-W64 project) 4.8.3, 64-bit
  Postgis 2.4.0dev - r15401 - 2017-05-25 19:11:42
  scripts 2.4.0dev r15401
  GEOS: 3.7.0dev-CAPI-1.11.0 8fe2ce6
  PROJ: Rel. 4.9.1, 04 March 2015
  SFCGAL: 1.3.0

tickets .. failed (diff expected obtained: /projects/postgis/tmp/2.4.0dev_pg10_geos3.7.0dev_gdal2.1.3w64/test_89_diff)
-----------------------------------------------------------------------------
--- tickets_expected	2016-09-26 01:59:38 -0400
+++ /projects/postgis/tmp/2.4.0dev_pg10_geos3.7.0dev_gdal2.1.3w64/test_89_out	2017-05-25 15:15:09 -0400
@@ -285,7 +285,7 @@
 #3375|GEOMETRYCOLLECTION(POINT(0 -7))
 #3399|1|1
 #3399|10|10
-#3399|100|100
+#3399|100|99
 #3399|1000|1000
 ERROR:  invalid KML representation
 #3437a|5
-----------------------------------------------------------------------------


  /var/lib/jenkins/workspace/postgis/branches/2.4/regress/00-regress-install/share/contrib/postgis/rtpostgis_proc_set_search_path.sql
PostgreSQL 10beta1 on x86_64-pc-linux-gnu, compiled by gcc (Debian 5.2.1-22) 5.2.1 20151010, 64-bit
  Postgis 2.4.0dev - r15401 - 2017-05-25 19:09:34
  scripts 2.4.0dev r15401
  raster scripts 2.4.0dev r15401
  GEOS: 3.7.0dev-CAPI-1.11.0 8fe2ce62
  PROJ: Rel. 4.9.2, 08 September 2015
  GDAL: GDAL 2.3.0dev, released 2017/99/99

Running tests

 check_gdal .. ok 
 load_outdb ... ok 
 check_raster_columns .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/test_3_diff)
-----------------------------------------------------------------------------
--- check_raster_columns_expected	2017-02-07 07:21:32.729359370 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/test_3_out	2017-05-25 19:14:07.797733794 +0000
@@ -1,23 +1,23 @@
 raster_columns
 t
-test_raster_columns|rast|0|1|1|2|2|t|f|1|{8BUI}|{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
+test_raster_columns|rast|0|1|1|||t|f|1|{8BUI}|{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
 t
 test_raster_columns|rast|0|||||f|f||||
 t
-test_raster_columns|rast|0|||2|2|f|f||||POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
+test_raster_columns|rast|0|||||f|f||||POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
 t
-test_raster_columns|rast|0|||2|2|f|f||||POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
+test_raster_columns|rast|0|||||f|f||||POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
 t
-test_raster_columns|rast|0|1|1|2|2|t|f|1||{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
+test_raster_columns|rast|0|1|1|||t|f|1||{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
 t
-test_raster_columns|rast|0|||2|2|t|f|1||{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
+test_raster_columns|rast|0|||||t|f|1||{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
 t
 t
 t
-test_raster_columns|rast|0|1|1|3|3|t|t|1|{8BUI}|{0}|POLYGON((0 0,0 6,6 6,6 0,0 0))
+test_raster_columns|rast|0|1|1|||t|t|1|{8BUI}|{0}|POLYGON((0 0,0 6,6 6,6 0,0 0))
 ERROR:  conflicting key value violates exclusion constraint "enforce_spatially_unique_test_raster_columns_rast"
 ERROR:  new row for relation "test_raster_columns" violates check constraint "enforce_coverage_tile_rast"
 t
-test_raster_columns|rast|0|1|1|3|3|t|f|1|{8BUI}|{0}|POLYGON((0 0,0 6,6 6,6 0,0 0))
+test_raster_columns|rast|0|1|1|||t|f|1|{8BUI}|{0}|POLYGON((0 0,0 6,6 6,6 0,0 0))
 t
 t
-----------------------------------------------------------------------------
 check_raster_overviews .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/test_4_diff)
-----------------------------------------------------------------------------
--- check_raster_overviews_expected	2017-02-07 07:21:33.465359381 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/test_4_out	2017-05-25 19:14:08.401733804 +0000
@@ -2,14 +2,14 @@
 test_raster_columns|rast|0|||||f|f||||
 test_raster_overviews|rast|0|||||f|f||||
 t
-test_raster_columns|rast|0|1|1|2|2|t|f|1|{8BUI}|{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
+test_raster_columns|rast|0|1|1|||t|f|1|{8BUI}|{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
 test_raster_overviews|rast|0|||||f|f||||
 t
-test_raster_columns|rast|0|1|1|2|2|t|f|1|{8BUI}|{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
+test_raster_columns|rast|0|1|1|||t|f|1|{8BUI}|{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
 test_raster_overviews|rast|0|||||f|f||||
 test_raster_overviews|rast|test_raster_columns|rast|1
 t
-test_raster_columns|rast|0|1|1|2|2|t|f|1|{8BUI}|{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
+test_raster_columns|rast|0|1|1|||t|f|1|{8BUI}|{0}|POLYGON((-2 -2,-2 4,4 4,4 -2,-2 -2))
 test_raster_overviews|rast|0|||||f|f||||
 t
 test_raster_columns|rast|0|||||f|f||||
-----------------------------------------------------------------------------
 rt_io .. ok 
 rt_bytea .. ok 
 box3d .. ok 
 rt_addband .. ok 
 rt_band .. ok 
 rt_tile .. ok 
 rt_dimensions .. ok 
 rt_scale .. ok 
 rt_pixelsize .. ok 
 rt_upperleft .. ok 
 rt_rotation .. ok 
 rt_georeference .. ok 
 rt_set_properties .. ok 
 rt_isempty .. ok 
 rt_hasnoband .. ok 
 rt_metadata .. ok 
 rt_rastertoworldcoord .. ok 
 rt_worldtorastercoord .. ok 
 rt_convexhull .. ok 
 rt_envelope .. ok 
 rt_band_properties .. ok 
 rt_set_band_properties .. ok 
 rt_pixelaspolygons .. ok 
 rt_pixelaspoints .. ok 
 rt_pixelascentroids .. ok 
 rt_setvalues_array .. ok 
 rt_summarystats .. ok 
 rt_count .. ok 
 rt_histogram .. ok 
 rt_quantile .. ok 
 rt_valuecount .. ok 
 rt_valuepercent .. ok 
 rt_bandmetadata .. ok 
 rt_pixelvalue .. ok 
 rt_neighborhood .. ok 
 rt_nearestvalue .. ok 
 rt_pixelofvalue .. ok 
 rt_polygon .. ok 
 rt_utility .. ok 
 rt_fromgdalraster .. ok 
 rt_asgdalraster .. ok 
 rt_astiff .. ok 
 rt_asjpeg .. ok 
 rt_aspng .. ok 
 rt_reclass .. ok 
 rt_gdalwarp .. ok 
 rt_asraster .. ok 
 rt_dumpvalues .. ok 
 rt_createoverview .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/test_53_diff)
-----------------------------------------------------------------------------
--- rt_createoverview_expected	2017-02-07 07:21:33.461359381 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/test_53_out	2017-05-25 19:14:20.769734015 +0000
@@ -1,29 +1,23 @@
 t
-t
-t
-t
-t
-res1|r|0|1|-1|10|10|t|t|t|t|t|t
-o_2_res1|r|0|2|-2|10|10|t|t|t|t|t|t
-o_4_res1|r|0|4|-4|10|10|t|t|t|t|t|t
-o_8_res1|r|0|8|-8|10|10|t|t|t|t|t|t
-o_16_res1|r|0|16|-16|10|10|t|t|t|t|t|t
-o_2_res1|r|res1|r|2
-o_4_res1|r|res1|r|4
-o_8_res1|r|res1|r|8
-o_16_res1|r|res1|r|16
-count|544|136|36|10|3
-t
-t
-t
-t
-t
-oschm|res1|r|0|1|-1|10|10|t|t|t|t|t|t
-oschm|o_4_res1|r|0|4|-4|10|10|t|t|t|t|t|t
-oschm|o_8_res1|r|0|8|-8|10|10|t|t|t|t|t|t
-public|res1|r|0|1|-1|10|10|t|f|f|f|t|f
-public|o_4_res1|r|0|4|-4|10|10|t|f|f|f|t|f
-public|o_8_res1|r|0|8|-8|10|10|t|f|f|f|t|f
-oschm|o_4_res1|r|oschm|res1|r|4
-oschm|o_8_res1|r|oschm|res1|r|8
-count|324|25|9
+ERROR:  cannot create overview without tilesize constraint, try select AddRasterConstraints('res1', 'r');
+ERROR:  cannot create overview without tilesize constraint, try select AddRasterConstraints('res1', 'r');
+ERROR:  cannot create overview without tilesize constraint, try select AddRasterConstraints('res1', 'r');
+ERROR:  cannot create overview without tilesize constraint, try select AddRasterConstraints('res1', 'r');
+res1|r|0|1|-1|||t|t|t|t|t|t
+ERROR:  relation "o_2_res1" does not exist at character 74
+ERROR:  table "o_16_res1" does not exist
+ERROR:  table "o_8_res1" does not exist
+ERROR:  table "o_4_res1" does not exist
+ERROR:  table "o_2_res1" does not exist
+t
+ERROR:  cannot create overview without tilesize constraint, try select AddRasterConstraints('oschm.res1', 'r');
+ERROR:  cannot create overview without tilesize constraint, try select AddRasterConstraints('oschm.res1', 'r');
+ERROR:  cannot create overview without tilesize constraint, try select AddRasterConstraints('res1', 'r');
+ERROR:  cannot create overview without tilesize constraint, try select AddRasterConstraints('res1', 'r');
+oschm|res1|r|0|1|-1|||t|t|t|t|t|t
+public|res1|r|0|1|-1|||t|f|f|f|t|f
+ERROR:  relation "oschm.o_4_res1" does not exist at character 80
+ERROR:  table "o_8_res1" does not exist
+ERROR:  table "o_4_res1" does not exist
+ERROR:  table "o_8_res1" does not exist
+ERROR:  table "o_4_res1" does not exist
-----------------------------------------------------------------------------
 rt_mapalgebraexpr .. ok 
 rt_mapalgebrafct .. ok 
 rt_mapalgebraexpr_2raster .. ok 
 rt_mapalgebrafct_2raster .. ok 
 rt_mapalgebrafctngb .. ok 
 rt_mapalgebrafctngb_userfunc .. ok 
 rt_intersection .. ok 
 rt_clip .. ok 
 rt_mapalgebra .. ok 
 rt_mapalgebra_expr .. ok 
 rt_mapalgebra_mask .. ok 
 rt_union .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/test_65_diff)
-----------------------------------------------------------------------------
--- rt_union_expected	2017-02-07 07:21:33.425359382 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/test_65_out	2017-05-25 19:14:24.161734073 +0000
@@ -53,9 +53,6 @@
 SUM|2|3|2
 SUM|3|3|2
 NOTICE:  No pixels found for band 1
-NOTICE:  No pixels found for band 1
-NOTICE:  No pixels found for band 1
-NOTICE:  No pixels found for band 1
 COUNT|1|1|1
 COUNT|2|1|1
 COUNT|3|1|1
-----------------------------------------------------------------------------
 rt_invdistweight4ma .. ok 
 rt_4ma .. ok 
 rt_setvalues_geomval .. ok 
 rt_elevation_functions .. ok 
 rt_colormap .. ok 
 rt_gist_relationships .. ok 
 rt_intersects .. ok 
 rt_samealignment .. ok 
 rt_geos_relationships .. ok 
 rt_iscoveragetile .. ok 
 bug_test_car5 .. ok 
 permitted_gdal_drivers .. ok 
 tickets .. ok 
 loader/Basic .... failed (testdiff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/test_79_diff)
-----------------------------------------------------------------------------
--- loader/Basic.select.expected	2017-02-07 07:21:32.889359372 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/test_79_out	2017-05-25 19:14:28.081734140 +0000
@@ -1,4 +1,4 @@
-0|1.0000000000|-1.0000000000|90|50|t|f|3|{8BUI,8BUI,8BUI}|{NULL,NULL,NULL}|{f,f,f}|POLYGON((0 -50,0 0,90 0,90 -50,0 -50))
+0|1.0000000000|-1.0000000000|||t|f|3|{8BUI,8BUI,8BUI}|{NULL,NULL,NULL}|{f,f,f}|POLYGON((0 -50,0 0,90 0,90 -50,0 -50))
 POLYGON((0 0,1 0,1 -1,0 -1,0 0))|255
 POLYGON((89 -49,90 -49,90 -50,89 -50,89 -49))|0
 POLYGON((44 -24,45 -24,45 -25,44 -25,44 -24))|0
-----------------------------------------------------------------------------
 loader/Projected .... failed ( test: running raster2pgsql output: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.err)
-----------------------------------------------------------------------------
BEGIN
psql:/var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.out:2: ERROR:  relation "loadedrast" already exists
-----------------------------------------------------------------------------
 failed (Errors while running sql script loader/Projected-post.sql: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/test_80_tmp)
-----------------------------------------------------------------------------
TRUNCATE TABLE
ERROR:  table "o_2_loadedrast" does not exist
-----------------------------------------------------------------------------
 ... but cleanup sql failed! loader/BasicCopy ... failed ( test: running raster2pgsql output: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.err)
-----------------------------------------------------------------------------
BEGIN
psql:/var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.out:2: ERROR:  relation "loadedrast" already exists
-----------------------------------------------------------------------------
 loader/BasicFilename ... failed ( test: running raster2pgsql output: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.err)
-----------------------------------------------------------------------------
BEGIN
psql:/var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.out:2: ERROR:  relation "loadedrast" already exists
-----------------------------------------------------------------------------
 loader/BasicOutDB ... failed ( test: running raster2pgsql output: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.err)
-----------------------------------------------------------------------------
BEGIN
psql:/var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.out:2: ERROR:  relation "loadedrast" already exists
-----------------------------------------------------------------------------
 loader/Tiled10x10 ... failed ( test: running raster2pgsql output: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.err)
-----------------------------------------------------------------------------
BEGIN
psql:/var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.out:2: ERROR:  relation "loadedrast" already exists
-----------------------------------------------------------------------------
 loader/Tiled10x10Copy ... failed ( test: running raster2pgsql output: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.err)
-----------------------------------------------------------------------------
BEGIN
psql:/var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.out:2: ERROR:  relation "loadedrast" already exists
-----------------------------------------------------------------------------
 loader/Tiled8x8 ... failed ( test: running raster2pgsql output: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.err)
-----------------------------------------------------------------------------
BEGIN
psql:/var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/loader.out:2: ERROR:  relation "loadedrast" already exists
-----------------------------------------------------------------------------
 clean .. ok 
 uninstall .  /var/lib/jenkins/workspace/postgis/branches/2.4/regress/00-regress-install/share/contrib/postgis/uninstall_rtpostgis.sql
 failed (Error encountered loading /var/lib/jenkins/workspace/postgis/branches/2.4/regress/00-regress-install/share/contrib/postgis/uninstall_rtpostgis.sql: /var/lib/jenkins/workspace/postgis/tmp/2_4_pg10.0w64/regress_log)
-----------------------------------------------------------------------------
sh: 1: createlang: not found

Change History (10)

comment:1 by robe, 7 years ago

Just tested on debbie's 10beta1 and raster seems okay but get same error on geometry: 

tickets .. failed (diff expected obtained: /projects/postgis/tmp/2.4.0dev_pg10_geos3.7.0dev_gdal2.1.3w64/test_89_diff)
-----------------------------------------------------------------------------
--- tickets_expected	2016-09-26 01:59:38 -0400
+++ /projects/postgis/tmp/2.4.0dev_pg10_geos3.7.0dev_gdal2.1.3w64/test_89_out	2017-05-25 15:15:09 -0400
@@ -285,7 +285,7 @@
 #3375|GEOMETRYCOLLECTION(POINT(0 -7))
 #3399|1|1
 #3399|10|10
-#3399|100|100
+#3399|100|99
 #3399|1000|1000
 ERROR:  invalid KML representation
 #3437a|5
-----------------------------------------------------------------------------

On my windows box, I'm getting the same errors as winnie with raster. So something has changed, and I'm pretty sure I tested this after I committed fix and it was only regress selectivity failing.

So I think something about r15401 caused this.

comment:2 by robe, 7 years ago

Aha smoking gun at least for the postgis regression. The raster regression might always have been there wasn't seen because of the selectivity failure might have been preventing raster tests.

the geometry regression in tickets caused by this change 

 nd_stats = palloc(sizeof(float) * sslot.nnumbers);
 memcpy(nd_stats, sslot.numbers, sizeof(float) * sslot.nnumbers);

To: 

 nd_stats = palloc(sizeof(float4) * sslot.nnumbers);
 memcpy(nd_stats, sslot.numbers, sizeof(float4) * sslot.nnumbers);

@pramsey - this change was not documented in your commit notes, was there a reason for it?

comment:3 by robe, 7 years ago

Okay I think the raster issue might be bigger than windows. Debbie is not getting to the raster tests because of the geometry failure. I'm seeing it on my local, since geometry tests pass after I roll back the float/float4 change.

comment:4 by pramsey, 7 years ago

That was a side change since sslot.numbers is defined as *float4, it seemed like allocating space based on sizeof(float4) seemed like a smart thing to do, platformwise. Disappointing that it's causing a regression, though potentially it's an improvement? Have to check and see what result we actually like better.

comment:5 by robe, 7 years ago

Component: buildbots → postgis
Owner: changed from robe to pramsey

comment:6 by robe, 7 years ago

okay at a glance the raster failures seem to be triggered by a possible behavior change in PostgreSQL 10 in exclusion constraints.

comment:7 by robe, 7 years ago

stand corrected. Debbie is exhibiting the same raster failures so not windows specific but definitely PostgreSQL 10 specific.

I suspect the rt_union failure is harmless and just result of behavior change with set returning functions.

comment:8 by robe, 7 years ago

okeedok, was mistaken about cause of raster failure being exclusion constraints, most are because of behavior change in PostgreSQL 10 CASE when, so all were fixed by r15404 (##3760 ) except for rt_union which I'll probably need to change the tests for.

comment:9 by robe, 7 years ago

Description: modified (diff)
Summary: PostgreSQL 10 failure on winnie both raster and geometry → PostgreSQL 10 failure on geometry

comment:10 by robe, 7 years ago

Resolution: → fixed
Status: new → closed

as @pramsey pointed out it's not the float/float4 change. Mysteriously it is no longer a problem on my PostgreSQL 10 and its a mystery why it is not. I'll reopen if it happens again.

Note: See TracTickets for help on using tickets.