#5711 closed defect (fixed)
rounding error in gdalwarp
Reported by: | youri | Owned by: | warmerdam |
---|---|---|---|
Priority: | normal | Milestone: | 2.0.0 |
Component: | Algorithms | Version: | 1.11.1 |
Severity: | normal | Keywords: | gdalwarp |
Cc: |
Description
Hello,
There are some rounding errors for integer negative numbers in gdalwarpkernel.cpp. Current behavior: 0 for -1.5 to 0.5, -1 for -2.5 to -1.5, etc.
the easy fix is to homogenize the rounding as done in other parts of the file. i.e:
1931c1931 < *pbValue = (GByte)(0.5 + dfValue); ---
*pbValue = (GByte)floor(0.5 + dfValue);
2006c2006 < *piValue = (GInt16)(0.5 + dfAccumulator); ---
*piValue = (GInt16)floor(0.5 + dfAccumulator);
2016c2016 < *piValue = (GInt16)(0.5 + dfAccumulator / dfAccumulatorDivisor); ---
*piValue = (GInt16)floor(0.5 + dfAccumulator / dfAccumulatorDivisor);
2139c2139 < *pbValue = (GByte)(0.5 + dfValue); ---
*pbValue = (GByte)floor(0.5 + dfValue);
2928c2928 < *pbValue = (GByte)(0.5 + dfAccumulator); ---
*pbValue = (GByte)floor(0.5 + dfAccumulator);
3011c3011 < *piValue = (GInt16)(0.5 + dfAccumulator); ---
*piValue = (GInt16)floor(0.5 + dfAccumulator);
(done also for GByte in the case if one day the signed byte type works)
Change History (2)
comment:1 by , 9 years ago
Milestone: | → 2.0 |
---|---|
Resolution: | → fixed |
Status: | new → closed |
Warping code has been significantly refactored in trunk and in particular there have been rounding fixes so I think this issue is fixed now.