I tracked down a bug while attempting to open a 2.7GB image file using a 64-bit linux OS.
The seg fault occurs during a memcpy when the pointer address becomes negative do to an integer overflow during a multiply/add.
The problem is the type (int) that is used in the multiply for iBufYOff * nLineSpace is not sufficient to hold a large enough number to point to the required memory address offset. Casting it to type (size_t) prior to the multiplication ensures the resulting type is large enough for the memory address space of the machine you are working on.
Here is the original and suggested code change.
root/trunk/gdal/gcore/rasterio.cpp
Before:
109 if( eRWFlag == GF_Read )
110 memcpy( ((GByte *) pData) + iBufYOff * nLineSpace,
111 pabySrcBlock + nSrcByteOffset,
112 nLineSpace );
113 else
114 memcpy( pabySrcBlock + nSrcByteOffset,
115 ((GByte *) pData) + iBufYOff * nLineSpace,
116 nLineSpace );
After:
109 if( eRWFlag == GF_Read )
110 memcpy( ((GByte *) pData) + ((size_t)iBufYOff * (size_t)nLineSpace),
111 pabySrcBlock + nSrcByteOffset,
112 nLineSpace );
113 else
114 memcpy( pabySrcBlock + nSrcByteOffset,
115 ((GByte *) pData) + ((size_t)iBufYOff * ((size_t)nLineSpace),
116 (size_t)nLineSpace );
