crash in mapresample.c with extreme coordinates

[warmerdam]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 47782147863200 (LWP 21954)]
0x0000000000469b42 in msNearestRasterResampler (psSrcImage=0x5a9efc0, offsite=
     {pen = -4, red = -1, green = -1, blue = -1}, psDstImage=0x5b04b70,
   panCMap=0x7fff87756710, pfnTransform=0x46c66a
<msApproxTransformer>, pCBData=0x597f8b0,
   debug=0) at mapresample.c:156
156                         int nValue = srcImg->tpixels[nSrcY][nSrcX];
(gdb) bt
#0  0x0000000000469b42 in msNearestRasterResampler
(psSrcImage=0x5a9efc0, offsite=
     {pen = -4, red = -1, green = -1, blue = -1}, psDstImage=0x5b04b70,
   panCMap=0x7fff87756710, pfnTransform=0x46c66a
<msApproxTransformer>, pCBData=0x597f8b0,
   debug=0) at mapresample.c:156
#1  0x000000000046e792 in msResampleGDALToMap (map=0x56f4280, layer=0x56fbae0,
   image=0x5b04b70, hDS=0x5c209f0) at mapresample.c:1654

local code:

            nSrcX = (int) x[nDstX];
            nSrcY = (int) y[nDstX];

            /*
             * We test the original floating point values to 
             * avoid errors related to asymmetric rounding around zero.
             */
            if( x[nDstX] < 0.0 || y[nDstX] < 0.0
                || nSrcX >= nSrcXSize || nSrcY >= nSrcYSize )
            {
                continue;
            }
...
                    int nValue = srcImg->tpixels[nSrcY][nSrcX];

It turns out that y[nDstX] was a very large positive value (larger than INT_MAX), but nSrcY became negative due to int overflow/wrap. The result is that y[nDstX] was not < 0 and nSrcY was not greater than nSrcYSize even though nSrcY was not suitable to use to subscript into the pixels array.

The solution appears to be an *almost* redundant check in nSrcX, and nSrcY.

[warmerdam]

Preliminary patch applied in trunk (r9296). This could be backported, but might bear some more testing before that is done.

[warmerdam]

Closing, we can revisit the backporting issue if someone wants, but the fix will be in 5.6.