VRT warped dataset crashes

[martin.daly@…]

We are getting two crashes with VRT warped datasets.  I have reproduced them in
the attached gdalinfo.c, with modifications pre- and post-marked with /* MPD */.

The sample data that we are using is from:

ftp://ftp.spotimage.fr/pub/dimap/S2_4DIM/S2_X_DIM.zip (40852 KB)

After unzipping, there are two folders, SCENE01 and SCENE02, both containing
IMAGERY.TIF files.  If you run the modified gdalinfo on the IMAGERY.TIF in
SCENE01, with the -mm flag, then you'll see the first crash in the extra call to
GDALComputeRasterMinMax.  After you fix that, you'll see the second in the added
call to GDALGetRandomRasterSample.

[martin.daly@…]

Modified gdalinfo.c for reproducing the bug

[warmerdam]

I have confirmed this bug.  It only manifests with relatively small
cache sizes (ie. --config GDAL_CACHEMAX 8).  It appears that the block
flushing logic is somehow getting confused and flushing the wrong block 
... one that is still locked (in use). 

I will track further.

[warmerdam]

This turned out to be a pretty esoteric set of interactions between low
level block handling and the vrt warped dataset. 

It seems that GDALRasterBand::GetBlockRef() was creating a block to read
into, and it was registered with the GDALRasterBlock "cache listing". 
However, AdoptBlock() was not called on it before the read.  AdoptBlock()
pushes the block into the block matrix for that band.  

Next VRTWarpedRasterBand::IReadBlock() called GetBlockRef() on the band 
in order to lock the block that it would be reading into before doing the
actual read process which might involve a great deal of io operations that
could have resulted in the flushing of the block being read into.  
However, because the block at the higher level had not yet been adopted, 
the VRTWarpedRasterBand() ended up causing a new block to get created and
adopted for the same "slot" in the band's block matrix. 

Later the top level block gets a flush request because it was never locked
but the block actually flushed is the one that was already adopted. 

The fundamental issue is uncertainty about whether low level drivers
are responsible for locking their blocks if they now their own operations
could result in the block being flushed. 

My correction is to change GDALRasterBand::GetBlockRef() so that the block 
is adopted before it is read into.  I also lock it so that any driver reader
that could trigger a flush will be prevented from flushing the block being
operated on.  

I have not removed the VRTWarpedRasterBand::IReadBlock() "locking" operation
since it might conceivably have an application in some other call situation. 
However, the higher level changes would seem to correct this problem for
all cases. 

The changes were applied in gdalrasterband.cpp (the GetBlockRef() method).

I believe the reported bug will only affect virtual warped raster access
in situations where the cache settings is relatively tight though the fix
may also avoid some bugs in other VRT cases where cache settings are tight.