Integer overflows in rasterfill.cpp
|Reported by:||akhliustov||Owned by:||warmerdam|
Description (last modified by )
if( iY - panLastY[iX] <= dfMaxSearchDist )
panLastY[iX] is unsigned, so
iY - panLastY[iX] is unsigned too. If
iY < panLastY[iX], their "difference" is a large positive number that can be greater than
dfMaxSearchDist. In that case the condition evaluates to
false, which is (probably) not expected.
double dfDistSq = ((target_x-origin_x) * (target_x-origin_x)) + ((target_y-origin_y) * (target_y-origin_y));
The right-hand expression here can have overflow on small enough input (I will try to attach it ASAP).
Change History (6)
comment:1 Changed 4 years ago by
|Summary:||Implicit signed-to-unsigned conversions in rasterfill.cpp → Integer overflows in rasterfill.cpp|