Opened 8 years ago
Closed 8 years ago
#6226 closed defect (fixed)
Excessive memory consumptions / leaks in block cache (GDAL 2.0)
| Reported by: | Even Rouault | Owned by: | Even Rouault |
|---|---|---|---|
| Priority: | normal | Milestone: | 2.0.2 |
| Component: | GDAL_Raster | Version: | 2.0.0 |
| Severity: | normal | Keywords: | |
| Cc: |
Description
There are two flows in the block cache of GDAL 2.0 :
- major one: an attempt is made to recycle the pData member of blocks that are going to be discarded, for reuse by the new allocated block. However currently we checked that the size of the to be discarded block was greater or equal than what is needed. The issue is that if the size was actually greater, then the counter of the block cache that tracks the total memory usage didn't see that. The memory wasn't leaked strictly speaking, but this could lead to excessive memory usage, disk trashing, out of memory, particularly when dealing with blocks of different sizes. For example converting a JPEG2000 with 2048x2048 tiles to a tiled TIFF with blocks of 256x256
- probably minor one: if the block cache memory was saturated, we tried to free at most 64 blocks. For some access patterns, this could potentially be a problem.
Note:
See TracTickets
for help on using tickets.
trunk r31656, branches/2.0 r31657 "Block cache: fix excessive memory consumption when dealing with datasets with different block sizes (#6226)"