MEM Driver does not support images larger than 4GB

[sy@…]

With the increasing use of 64-bit machines, in-memory images of larger than 4GB are now in theory possible. However the GDAL MEM driver needs to be upgraded to support this, perhaps by using the size_t type which is guaranteed to be large enough to support any memory offset on a given platform.

[warmerdam]

I was able to reproduce a bug with large arrays using the following program, and
I have committed a fix for this problem in trunk and 1.4 branch that seems to
fix it. 

#include "gdal.h"
#include "cpl_conv.h"

/************************************************************************/
/*                                main()                                */
/************************************************************************/
int main()

{
    size_t nMemSize = 5000000000;
    GByte *pabyBigMem = (GByte *) calloc(1,nMemSize);

/* -------------------------------------------------------------------- */
/*      Setup memory view of array.                                     */
/* -------------------------------------------------------------------- */
    GDALAllRegister();

    GDALDatasetH hDS = GDALCreate( GDALGetDriverByName("MEM"),
                                   "", 50000, 50000, 0, GDT_Byte, NULL );
    
    char szDataPointer[100];
    char szPixelOffset[100];
    char szLineOffset[100];

    sprintf( szDataPointer, "DATAPOINTER=%p", pabyBigMem );
    sprintf( szPixelOffset, "PIXELOFFSET=2" );
    sprintf( szLineOffset, "LINEOFFSET=100000" );

    char *apszOptions[4];
    
    apszOptions[0] = szDataPointer;
    apszOptions[1] = szPixelOffset;
    apszOptions[2] = szLineOffset;
    apszOptions[3] = NULL;
    
    GDALAddBand( hDS, GDT_Byte, apszOptions );

    sprintf( szDataPointer, "DATAPOINTER=%p", pabyBigMem+1 );
    GDALAddBand( hDS, GDT_Byte, apszOptions );

/* -------------------------------------------------------------------- */
/*      Set the last two bytes of the buffer, and verify they read      */
/*      back as the bottom right pixels of the image.                   */
/* -------------------------------------------------------------------- */

    pabyBigMem[nMemSize-2] = 128;
    pabyBigMem[nMemSize-1] = 255;

    GByte abyResult[2];
    GDALDatasetRasterIO( hDS, GF_Read, 49999, 49999, 1, 1, 
                         abyResult, 1, 1, GDT_Byte, 
                         2, NULL, 0, 0, 0 );
    if( abyResult[0] != 128 || abyResult[1] != 255 )
    {
        printf( "Got wrong result: %d/%d\n", abyResult[0], abyResult[1] );
    }
    else
    {
        printf( "Result OK!\n" );
    }

    GDALClose( hDS );
    free( pabyBigMem );
}