Opened 6 years ago

Closed 6 years ago

Last modified 5 years ago

#2516 closed defect (fixed)

[raster] Debbie failing on GDAL_Trunk PostGIS regress

Reported by: robe Owned by: Bborie Park
Priority: critical Milestone: PostGIS GDAL
Component: raster Version: trunk
Keywords: Cc: warmerdam

Description

Debbie seems to be failing now on some regress tests after GDAL trunk update

http://debbie.postgis.net:8080/job/GDAL_Trunk/1115/
GDAL SVN Revision 26554
    added DST_METHOD and support for GCP and TPS dest
    added DST_METHOD testing for GCP_POLYNOMIAL and GCP_TPS

which changed /trunk/gdal/alg/gdaltransformer.cpp

The tests failing:

Fulll ong here with errors http://debbie.postgis.net:8080/job/GDAL_PostGIS_Regress/1051/console

abbreviated and a crasher at rt_setvalues_geomval_expected +psql: FATAL: the database system is in recovery mode

PostgreSQL 9.2.4 on x86_64-unknown-linux-gnu, compiled by gcc (Debian 4.4.5-8) 4.4.5, 64-bit
  Postgis 2.2.0dev - r12054 - 2013-10-26 11:22:42
  GEOS: 3.5.0dev-CAPI-1.9.0 r3883
  PROJ: Rel. 4.7.1, 23 September 2009
  GDAL: GDAL 1.11dev, released 2013/04/13

Running tests

 check_gdal .. ok 
 load_outdb ... ok 
 check_raster_columns .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_3_diff)
 check_raster_overviews .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_4_diff)
 rt_io .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_5_diff)
 rt_bytea .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_6_diff)
 box3d .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_7_diff)
 rt_addband .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_8_diff)
 rt_band .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_9_diff)
 rt_tile .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_10_diff)
 rt_dimensions .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_11_diff)
 rt_scale .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_12_diff)
 rt_pixelsize .. ok 
 rt_upperleft .. ok 
 rt_rotation .. ok 
 rt_georeference .. ok 
 rt_set_properties .. ok 
 rt_isempty .. ok 
 rt_hasnoband .. ok 
 rt_metadata .. ok 
 rt_rastertoworldcoord .. ok 
 rt_worldtorastercoord .. ok 
 rt_convexhull .. ok 
 rt_band_properties .. ok 
 rt_set_band_properties .. ok 
 rt_pixelaspolygons .. ok 
 rt_pixelaspoints .. ok 
 rt_pixelascentroids .. ok 
 rt_setvalues_array .. ok 
 rt_summarystats .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_30_diff)
 rt_count .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_31_diff)
 rt_histogram .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_32_diff)
 rt_quantile .. ok 
 rt_valuecount .. ok 
 rt_valuepercent .. ok 
 rt_bandmetadata .. ok 
 rt_pixelvalue .. ok 
 rt_neighborhood .. ok 
 rt_nearestvalue .. ok 
 rt_pixelofvalue .. ok 
 rt_polygon .. ok 
 rt_utility .. ok 
 rt_fromgdalraster .. ok 
 rt_asgdalraster .. ok 
 rt_astiff .. ok 
 rt_asjpeg .. ok 
 rt_aspng .. ok 
 rt_reclass .. ok 
 rt_gdalwarp .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_49_diff)
 rt_asraster .. ok 
 rt_dumpvalues .. ok 
 rt_mapalgebraexpr .. ok 
 rt_mapalgebrafct .. ok 
 rt_mapalgebraexpr_2raster .. ok 
 rt_mapalgebrafct_2raster .. ok 
 rt_mapalgebrafctngb .. ok 
 rt_mapalgebrafctngb_userfunc .. ok 
 rt_intersection .. ok 
 rt_clip .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_59_diff)
 rt_mapalgebra .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_60_diff)
 rt_mapalgebra_expr .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_61_diff)
 rt_union .. ok 
 rt_invdistweight4ma .. ok 
 rt_4ma .. ok 
 rt_setvalues_geomval .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_65_diff)
 rt_elevation_functions .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/gdal_regress/tmp/2_2_pg9.2w64/test_66_diff)
 rt_colormap .. ok 
 rt_gist_relationships .. ok 
 rt_intersects .. ok 
 rt_samealignment .. ok 
 rt_geos_relationships .. ok 
 rt_iscoveragetile .. ok 
 bug_test_car5 .. ok 
 tickets .. ok 
 loader/Basic .... ok 
 loader/BasicCopy .... ok 
 loader/BasicFilename .... ok 
 loader/BasicOutDB .... ok 
 loader/Tiled10x10 .... ok 
 loader/Tiled10x10Copy .... ok 
 loader/Tiled8x8 .... ok 
 clean .. ok 
 uninstall .

Change History (20)

comment:1 Changed 6 years ago by robe

Component: buildbotsraster
Owner: changed from robe to Bborie Park

comment:2 Changed 6 years ago by robe

Dusty or Pierre,

When you get a chance, can you try build PostGIS with latest GDAL. I'll try to do that locally but not setup to at moment, to rule this out as just a bot issue.

comment:3 Changed 6 years ago by robe

Here is the GDAL commit which I think triggered the issue

http://trac.osgeo.org/gdal/changeset/26554

comment:4 Changed 6 years ago by Bborie Park

Status: newassigned

Confirmed. Thanks for doing the leg-work... I'll see how fast I can fix this.

comment:5 Changed 6 years ago by Bborie Park

There has been lots of work on GDALCreateGenImgProjTransformer2() in the last day or two. I'll give it a few days in case the new GDAL behavior is to be expected.

comment:6 Changed 6 years ago by robe

Bborie,

For the record I've been chatting with EvenR on IRC. He told me to keep him abreast of the situation. It doesn't sound like they intentionally meant to break anything in PostGIS land. Maybe we can ask Frank about it.

Can we at least confirm this is a GDAL change that broke our code? Then maybe we can put in a separate ticket on GDAL to find out the intention.

comment:7 Changed 6 years ago by Bborie Park

Yes, the GDAL change resulted in different behavior. I'm going to see what changed exactly.

comment:8 Changed 6 years ago by rouault

I have reviewed the differences in GDALCreateGenImgProjTransformer2() and the use made of it in PostGIS Raster code (2 occurrences in in rt_warp.c). I believe that there should be no impact on the one at line 358, but there might be an influence on the one at line 840 if the geotransform on the dest ds is set to [ 0, 1, 0, 0, 0, +/- 1 ]. If that's the case, I believe that you get a "Unable to compute a transformation between pixel/line and georeferenced coordinates for ..." error message from GDAL, and a NULL result, that causes rt_raster_gdal_warp() to return in error.

Is it the case ?

comment:9 Changed 6 years ago by Bborie Park

Yes, that geotransform can be present in PostGIS. I was looking at the problem through GDALRasterizeGeometries() in rt_raster.c. The test geometry and raster I am using makes use of that geotransform [0, 1, 0, 0, 0, -1].

comment:10 Changed 6 years ago by warmerdam

Cc: warmerdam added

I'll remove the check for the null transform. I only added it for similarity to the rules on the "source" side of the transformation.

comment:11 Changed 6 years ago by warmerdam

OK, that change is now in gdal (r26578).

comment:12 Changed 6 years ago by Bborie Park

Thanks warmerdam.

I plan on testing what happens if non-spatially referenced input rasters and geometries are made to be spatially referenced (faked). If the tests work, you should be able to undo r26578.

comment:13 Changed 6 years ago by Bborie Park

Well, I couldn't get non-spatial geometries and rasters with the default geotransform [0, 1, 0, 0, 0, -1] to work with faked spatial references.

Thanks warmerdam!

comment:14 Changed 6 years ago by Bborie Park

Resolution: wontfix
Status: assignedclosed

I'm closing this as "wontfix" since there isn't an option "cantfix". The real resolution is for postgis to have a full-fledged rasterizer as per #1168.

comment:15 in reply to:  14 Changed 6 years ago by rouault

Replying to dustymugs:

I'm closing this as "wontfix" since there isn't an option "cantfix". The real resolution is for postgis to have a full-fledged rasterizer as per #1168.

Hum, I don't understand the link between rasterization and the issue raised in that ticket... Is there anything left on GDAL side related to gdaltransformer that would be needed for PostGIS raster ?

comment:16 Changed 6 years ago by Bborie Park

No, there is nothing left of the GDAL side related to gdaltransformer that needs to be done. I am concerned of GDAL being blocked (warmerdam having to undo some of his work) because of PostGIS, which is probably not the best thing for either project in the long term.

comment:17 Changed 6 years ago by rouault

Well, in the instance, it was a test that wasn't strictly necessary and doesn't remove functionnality in GDAL. Feedback (and contributions) from projects using GDAL are mutually valuable.

comment:18 Changed 6 years ago by robe

Milestone: PostGIS 2.2.0PostGIS GDAL
Resolution: wontfix
Status: closedreopened

Just to maintain accuracy of this ticket to repflect the issue was INDEED resolved and did not require a change in PostGIS, I've flipped this.

comment:19 Changed 6 years ago by robe

Resolution: fixed
Status: reopenedclosed

Was only an issue in latest GDAL trunk which Frank has resolved. Thank you Frank and Even for the quick response and resolution.

comment:20 Changed 5 years ago by strk

The problem of identity matrix was found also on the usage with no destination, see #2914

Note: See TracTickets for help on using tickets.