id summary reporter owner description type status priority milestone component version severity resolution keywords cc 2457 GDALRasterBand::IRasterIO fails to read data from dataset when block cache is too small Even Rouault warmerdam "To demonstrate the problem, run : {{{ GDAL_CACHEMAX=0 gdal_translate ../autotest/gcore/data/utmsmall.tif temp.tif -co TILED=YES -co BLOCKXSIZE=16 -co BLOCKYSIZE=16 }}} The resulting checksum will be wrong (46979) instead of 50054. In rasterio.cpp, we are in the case of line 171. The cache size is such that blocks get flushed out several times while iterating over iBufYOff, but when they are reloaded their content is not read back from the dataset. The GDAL_CACHEMAX=0 value is of course very particular, but I think we could also hit that case when writing buffers very large in the X direction, in fact if blockByteSize * (nXSize / nBlockXSize) > GDAL_CACHEMAX. So, if nBlockXSize = nBlockYSize = 256 and GDAL_CACHEMAX = 40 MB, we have the critical nXSize ~= 155 000 The following code also demonstrates the problem: {{{ import gdal one_ds = gdal.GetDriverByName(""GTiff"").Create(""ones.tif"", 200000, 256, 1, options = [ 'TILED=YES', 'BLOCKXSIZE=256', 'BLOCKYSIZE=256']) one_ds.GetRasterBand(1).Fill(1) ds = gdal.GetDriverByName(""GTiff"").Create(""temp.tif"", 200000, 256, 1, options = [ 'TILED=YES', 'BLOCKXSIZE=256', 'BLOCKYSIZE=256']) data = one_ds.ReadRaster(0, 0, 200000, 256) ds.WriteRaster(0, 0, 200000, 256, data) one_ds = None ds = None }}} In DEBUG mode, we have the following complaint : ""GDAL: Potential thrashing on band 1 of ones.tif"" The solution is to rearrange a bit the way we write the block such that we fill it up completely instead of loading it each time. That is to say that we have a sub-loop on y at the deeper level. This will save quite a few call to GetLockedBlockRef at the same time. " defect closed normal 1.5.3 GDAL_Raster 1.5.0 normal fixed