RFC 19: Safer memory allocation in GDAL

Author: Even Rouault
Contact: even.rouault@…
Status: Development

Summary

This document contains proposal on how to make GDAL safer (prevent crashes) when doing memory allocations. The starting point of this discussion is ticket #2075.

Details

In many places in GDAL source code, multiplications are done to compute the size of the memory buffer to allocate, like raster blocks, scanlines, whole image buffers, etc.. Currently no overflow checking is done, thus leading to potential allocation of not large enough buffers. Overflow can occur when raster dimensions are very large (this can be the case with a WMS raster source for example) or when a dataset is corrupted, intentionnaly or unintentionnaly. This can lead to latter crash.

This RFC introduces new API to do multiplications on size_t variables and report overflows when they occur. Overflows are detected by checking that ((a*b)/b) == a. This does not require to make assumptions on the size of the variable types, their signedness, etc.

/* The following 2 functions check that the result of the multiplication */
/* does not overflow the limits of size_t. 0 is returned in case of overflow */
/* *pbOverflowFlag is set to TRUE if overflow has occured */
size_t CPL_DLL VSISafeMul2( size_t mul1, size_t mul2, int *pbOverflowFlag);
size_t CPL_DLL VSISafeMul3( size_t mul1, size_t mul2, size_t mul3, int *pbOverflowFlag);

/* This function return the @size argument if it fits into an integer, or 0 in */
/* case of overflow. */
/* *pbOverflowFlag is set to TRUE if overflow has occured */
int    CPL_DLL VSISafeSizetCastToInt( size_t size, int *pbOverflowFlag );

Note: the pbOverflowFlag parameter is new in comparison to the initial proposition of ticket #2075 .

Implementation of CPLMalloc, CPLCalloc, CPLRealloc, VSIMalloc, VSICalloc, VSIRealloc will not be changed. Developers are encouraged to use the VSI functions rather than the CPL ones since CPLMalloc will abort the process when doing a too large allocation, whereas VSIMalloc will return a NULL pointer.

Implentation steps

1. Introduce the new API in gdal/port
   
2. Use the new API in GDAL core where it is relevant. The following files have been identified as candidates : gcore/gdalnodatamaskband.cpp, gcore/overview.cpp, gcore/gdaldriver.cpp, gcore/gdalrasterblock.cpp
   
3. Use the new API in GDAL drivers. This step can be done incrementally. Transition from CPL to VSI allocation can be necessary in some cases too. Candidate drivers : Idrisi, PNG, GXF, BSB, VRT, MEM, JP2KAK, RPFTOC, AIRSAIR, AIGRIB, XPM, USGDEM, BMP, GSG, HFA, AAIGRID. (See gdal_svn_trunk_use_vsi_safe_mul_in_frmts.patch in ticket #2075)

Even Rouault will implement the changes described in this RFC for the GDAL 1.6.0 release.

Voting history

Not submitted to vote