Opened 9 years ago

Closed 9 years ago

Last modified 9 years ago

#5711 closed defect (fixed)

rounding error in gdalwarp

Reported by: youri Owned by: warmerdam
Priority: normal Milestone: 2.0.0
Component: Algorithms Version: 1.11.1
Severity: normal Keywords: gdalwarp
Cc:

Description

Hello,

There are some rounding errors for integer negative numbers in gdalwarpkernel.cpp. Current behavior: 0 for -1.5 to 0.5, -1 for -2.5 to -1.5, etc.

the easy fix is to homogenize the rounding as done in other parts of the file. i.e:

1931c1931 < *pbValue = (GByte)(0.5 + dfValue); ---

*pbValue = (GByte)floor(0.5 + dfValue);

2006c2006 < *piValue = (GInt16)(0.5 + dfAccumulator); ---

*piValue = (GInt16)floor(0.5 + dfAccumulator);

2016c2016 < *piValue = (GInt16)(0.5 + dfAccumulator / dfAccumulatorDivisor); ---

*piValue = (GInt16)floor(0.5 + dfAccumulator / dfAccumulatorDivisor);

2139c2139 < *pbValue = (GByte)(0.5 + dfValue); ---

*pbValue = (GByte)floor(0.5 + dfValue);

2928c2928 < *pbValue = (GByte)(0.5 + dfAccumulator); ---

*pbValue = (GByte)floor(0.5 + dfAccumulator);

3011c3011 < *piValue = (GInt16)(0.5 + dfAccumulator); ---

*piValue = (GInt16)floor(0.5 + dfAccumulator);

(done also for GByte in the case if one day the signed byte type works)

Change History (2)

comment:1 by Even Rouault, 9 years ago

Milestone: 2.0
Resolution: fixed
Status: newclosed

Warping code has been significantly refactored in trunk and in particular there have been rounding fixes so I think this issue is fixed now.

comment:2 by Even Rouault, 9 years ago

Milestone: 2.02.0.0

Milestone renamed

Note: See TracTickets for help on using tickets.