Opened 11 years ago

Closed 11 years ago

Last modified 10 years ago

#3458 closed defect (fixed)

blank rectangles from gdalwarp for large inputs

Reported by: dstahlke Owned by: warmerdam
Priority: normal Milestone: 1.7.2
Component: Algorithms Version: 1.7.1
Severity: normal Keywords: warper
Cc:

Description (last modified by warmerdam)

gdalwarp is generating output that has large black rectangles that should not be there when given a large input file (a bluemarble octant). The rectangles move around depending on the output file format and depending on the value passed to the "-wm" option. If I use "-wm 1024" or greater the problem goes away (the input file is 1.3GB). The warp command I am using is:

gdalwarp -rn -te -4000000 -3000000 0 3000000 \
  -t_srs '+proj=gnom  +lon_0=-90.0 +lat_0=58 +a=5500000 +b=5500000' \
  -tr 2000 2000 -co 'TILED=YES' zz-src.tif zz.tif

The same error happens with other projections, and without the TILED=YES option. Valgrind doesn't report any errors.

I've attached a small thumbnail showing what it looks like.

Attachments (1)

zz.jpg (12.6 KB) - added by dstahlke 11 years ago.
thumbnail of output

Download all attachments as: .zip

Change History (8)

Changed 11 years ago by dstahlke

Attachment: zz.jpg added

thumbnail of output

comment:1 Changed 11 years ago by warmerdam

Component: defaultAlgorithms
Description: modified (diff)
Status: newassigned

Can you provide a gdalinfo report for the input file so I can try and construct something structurally similar to test?

comment:2 Changed 11 years ago by dstahlke

Here is the gdalinfo report. I could also try and upload it to rapidshare or something, but the file is big.

Driver: GTiff/GeoTIFF
Files: zz-src.tif
Size is 21600, 21600
Coordinate System is:
GEOGCS["unnamed ellipse",
    DATUM["unknown",
        SPHEROID["unnamed",6370997,0]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433]]
Origin = (-180.000000000000000,90.000000000000000)
Pixel Size = (0.004166666666667,-0.004166666666667)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (-180.0000000,  90.0000000) (180d 0'0.00"W, 90d 0'0.00"N)
Lower Left  (-180.0000000,   0.0000000) (180d 0'0.00"W,  0d 0'0.01"N)
Upper Right ( -90.0000000,  90.0000000) ( 90d 0'0.00"W, 90d 0'0.00"N)
Lower Right ( -90.0000000,   0.0000000) ( 90d 0'0.00"W,  0d 0'0.01"N)
Center      (-135.0000000,  45.0000000) (135d 0'0.00"W, 45d 0'0.00"N)
Band 1 Block=21600x1 Type=Byte, ColorInterp=Red
Band 2 Block=21600x1 Type=Byte, ColorInterp=Green
Band 3 Block=21600x1 Type=Byte, ColorInterp=Blue

comment:3 Changed 11 years ago by warmerdam

I'm getting exactly the same effect with trunk. It is not immediately obvious what is going wrong.

comment:4 Changed 11 years ago by warmerdam

It is interesting to note that the middle of image whole falls on tile boundaries, while the bottom omission does not appear to be on a tile boundary.

I see the last section processed is:

..50...60..GDAL: GDALWarpKernel()::GWKNearestNoMasksByte()
Src=5506,3291,5295x3869 Dst=768,1280,1232x1500
.70...80...90...100 - done.

so it may be that no effort is even made to produce the final section.

comment:5 Changed 11 years ago by warmerdam

Keywords: warper added
Milestone: 1.7.2
Resolution: fixed
Status: assignedclosed

It turns out the missing areas just aren't in the list of chunks that are processed. The problem is caused by the chunking codes recent attempts to operate on block boundaries, and an improper calculation of the remainder region.

GDAL: GDALWarpKernel()::GWKNearestNoMasksByte()
Src=634,2606,4873x2682 Dst=0,0,768x1280
0...10...GDAL: GDALWarpKernel()::GWKNearestNoMasksByte()
Src=1025,1604,3860x1489 Dst=768,0,512x512
20.GDAL: GDALWarpKernel()::GWKNearestNoMasksByte()
Src=1731,406,9069x1802 Dst=1280,0,720x512
..GDAL: GDALWarpKernel()::GWKNearestNoMasksByte()
Src=3202,1468,7599x2605 Dst=768,512,1232x640
30...40.GDAL: GDALWarpKernel()::GWKNearestNoMasksByte()
Src=3891,4303,3807x3248 Dst=0,1280,768x1500
..50...60..GDAL: GDALWarpKernel()::GWKNearestNoMasksByte()
Src=5506,3291,5295x3869 Dst=768,1280,1232x1500

I have corrected the problem in trunk (r19009) and 1.7 branch (r19010).

Note, I also found a problem with untiled output filed as #3459 which may be related to this, but is not fixed by this fix.

comment:6 Changed 11 years ago by dstahlke

Frank,

Thanks for the rapid fix!

comment:7 Changed 10 years ago by Even Rouault

r19274 /trunk/autotest/warp/warp.py: Add test for warping datasets which are 'bigger' than the wm parameter. Would have detected issue of #3458

Note: See TracTickets for help on using tickets.