GDALDriver::CreateCopy() and gdal_translate failed on large image
|Reported by:||ozys||Owned by:||warmerdam|
Data used: NITF, 16-bit, scanline, 40k x 100k panchromatic image, JPEG2000 compressed.
Trying to create a copy of above data with square/rectangular block-size.
CreateCopy?() memory use grows along with the size of image it tries to copy. With large image as above, CreateCopy?() was able to complete about 80% of its task when it used up all 8GB of swap and 98.5% of 8GB memory before the system killed the process. Similar trend is observed when using gdal_translate as well.
We also tested using uncompressed NITF, and the similar trend of growing memory use was observed as well.
valgrind was used to detect problems (gdal compiled with flags). No memory leak was found related to this issue. So it appears that CreateCopy?() has an ever growing request for memory which seems to correlate with the size of file it copies. This block of memory appears to be freed appropriately at the end of the copy process, thus valgrind does not see it as a leak.
We tested this using gdal 1.7 and the previous release of gdal with identical results. GDAL cache was varied as well from 64MB to 1GB with similar results (failure to complete the task and processed was killed by system).