id summary reporter owner description type status priority milestone component version severity resolution keywords cc 5054 [PATCH - libtiff] TIFFMergeFieldInfo error when adding overviews to large tiff EliL warmerdam "This ticket comes from the email [http://lists.osgeo.org/pipermail/gdal-dev/2013-April/035958.html list] gdaladdo topo_mosaic_4326.tif 2 4 8 16 32 64 128 256 512 1024 --config COMPRESS_OVERVIEW PACKBITS ERROR 1: topo_mosaic_4326.tif:Failed to allocate memory for for fields array (1410046 elements of 16 bytes each) >gdalinfo -checksum topo_mosaic_4326.tif Size is 398449, 210532 Even reported: ""I've investigated a bit and I've discovered that while computing overviews, libtiff constantly switches from TIFF ""directories"" (main image, first overview,second overview, etc...). While doing this, it calls each time client code to install its own TIFF tags (libgeotiff for the geotiff tags, GDAL for the GDAL specific TIFF tags), but doesn't reset its array of existing tags. So each time the tags are added again and again, hence the huge memory allocation."" This can be observed on 32-bit windows with a large file like: Size is 398449, 210532, or on linux if built with additional instrumented components: {{{ $ gdaladdo topo_mosaic_4326.tif 2 4 8 16 32 64 128 256 512 1024 --config COMPRESS_OVERVIEW PACKBITS 0...10...20...30...40...50...60...70...80...90...100 - done. Current VSI memory usage : 0 bytes Maximum VSI memory usage : 2361761086 bytes Number of calls to VSIMalloc() : 43981823 Number of calls to VSICalloc() : 1259 Number of calls to VSIRealloc() : 17378556 Number of calls to VSIFree() : 43983082 VSIMalloc + VSICalloc - VSIFree : 0 }}} " defect closed normal 2.0.0 GDAL_Raster svn-trunk normal fixed libtiff, tiff tags, overviews