Excessive memory consumptions / leaks in block cache (GDAL 2.0)
There are two flows in the block cache of GDAL 2.0 :
- major one: an attempt is made to recycle the pData member of blocks that are going to be discarded, for reuse by the new allocated block. However currently we checked that the size of the to be discarded block was greater or equal than what is needed. The issue is that if the size was actually greater, then the counter of the block cache that tracks the total memory usage didn't see that. The memory wasn't leaked strictly speaking, but this could lead to excessive memory usage, disk trashing, out of memory, particularly when dealing with blocks of different sizes. For example converting a JPEG2000 with 2048x2048 tiles to a tiled TIFF with blocks of 256x256
- probably minor one: if the block cache memory was saturated, we tried to free at most 64 blocks. For some access patterns, this could potentially be a problem.
Note: See TracTickets for help on using tickets.