Opened 7 years ago

Closed 7 years ago

#5874 closed defect (fixed)

Rounding error when calculating output window on VRTSimpleSource

Reported by: olavobelloc Owned by: warmerdam
Priority: highest Milestone: 1.11.3
Component: GDAL_Raster Version: 1.11.2
Severity: major Keywords: VRT VRTSimpleSource
Cc: antonio


In the file frmts/vrt/vrtsources.cpp, the function VRTSimpleSource::GetSrcDstWindow?(...) contains what seems to be a rounding error while calculating the output window into the output buffer.

The output offset and size are calculated at the end of this function, and are saved into the variables: pnOutXOff, pnOutYOff, pnOutXSize and pnOutYSize. The offset is truncated as expected, however, the size should be rounded appropriately, to ensure the last pixel in the output window might be covered as well.

The same logic is used when calculating the size of the request window, on the variables: pnReqXSize and pnReqYSize. However, the size of the request window is properly rounded by adding 0.5 to the floating point result before truncating. Which does NOT happen in the output window.

The fix I suggest is as simple as adding 0.5 before truncation of the pnOutXSize and pnOutYSize variables.

In the file frmts/vrt/vrtsources.cpp, just replace the lines that mention:

*pnOutXSize = (int) ((dfDstLRX - nXOff) * dfScaleWinToBufX+0.001) - *pnOutXOff;


*pnOutYSize = (int) ((dfDstLRY - nYOff) * dfScaleWinToBufY+0.001) - *pnOutYOff;

by these, respectively:

*pnOutXSize = (int) ((dfDstLRX - nXOff) * dfScaleWinToBufX+0.501) - *pnOutXOff;


*pnOutYSize = (int) ((dfDstLRY - nYOff) * dfScaleWinToBufY+0.501) - *pnOutYOff;

I have noticed this bug while using osgearth to raster a few hundreds of geotiffs on a terrain. The terrain tiles are calculated by rastering a GDAL VRT data set. The result contained a few tiles with a black pixel line, actually, the last line of the output window. After making the suggested changes above, the black lines just disappeared, and everything looks just nice now. =)

Change History (2)

comment:1 Changed 7 years ago by antonio

Cc: antonio added

comment:2 Changed 7 years ago by Even Rouault

Milestone: 1.11.3
Resolution: fixed
Status: newclosed

trunk r28734, branches/1.11 r28735 "VRT: fix rounding of output window size on VRTSimpleSource (patch by olavobelloc, #5874)"

Note: See TracTickets for help on using tickets.