Changes between Initial Version and Version 1 of Ticket #5203
- Timestamp:
- Aug 18, 2013, 9:14:40 PM (11 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Ticket #5203
- Property Summary Implicit signed-to-unsigned conversions in rasterfill.cpp → Integer overflows in rasterfill.cpp
-
Ticket #5203 – Description
initial v1 2 2 {{{if( iY - panLastY[iX] <= dfMaxSearchDist )}}} [[BR]][[BR]] {{{panLastY[iX]}}} is unsigned, so {{{iY - panLastY[iX]}}} is unsigned too. If {{{iY < panLastY[iX]}}}, their "difference" is a large positive number that can be greater than {{{dfMaxSearchDist}}}. In that case the condition evaluates to {{{false}}}, which is (probably) not expected. 3 3 4 2. http://trac.osgeo.org/gdal/browser/trunk/gdal/alg/rasterfill.cpp#L333 [[BR]][[BR]] {{{double dfDistSq = ((target_x-origin_x) * (target_x-origin_x)) + ((target_y-origin_y) * (target_y-origin_y));}}} [[BR]][[BR]] Here {{{target_y}}} is unsigned, so the whole right-hand expression uses unsigned arithmetics. If {{{target_x < origin_x}}} or {{{target_y < origin_y}}}, it leads to a totally wrong result.4 2. http://trac.osgeo.org/gdal/browser/trunk/gdal/alg/rasterfill.cpp#L333 [[BR]][[BR]] {{{double dfDistSq = ((target_x-origin_x) * (target_x-origin_x)) + ((target_y-origin_y) * (target_y-origin_y));}}} [[BR]][[BR]] The right-hand expression here can have overflow on small enough input (I will try to attach it ASAP).