Erronous size output for Gtiff using gdalwarp

[jrm]

Hi,

I'm encountering a tiff's output size issue with gdalwarp when trying to reproject a file using a grid transformation from LAMB1 (EPSG:27571) to LAMB93 (EPSG:2154), the operation cannot succeed because the process is taking more disk space than available. By too much I mean over 50Go while it should takes less than 4Go (based on a gdal_translate's test, see below).

The source is file01.ecw = 680Mo for 48934x46280

gdal_translate -of Gtiff -co "BIGTIFF=YES" -co "INTERLEAVE=PIXEL" -co "PROFILE=GDALGeoTIFF" -co "TFW=YES" -co "COMPRESS=LZW" "file01.ecw" "file02.tif" 

file02.tif = 3.8Go

gdalwarp -s_srs "+init=IGNF:LAMB1C +wktext" -t_srs "+init=IGNF:LAMB93 +wktext" -rc -tr 2.5 2.5 -multi -of Gtiff -co "BIGTIFF=YES" -co "INTERLEAVE=PIXEL" -co "PROFILE=GDALGeoTIFF" -co "TFW=YES" -co "COMPRESS=LZW" -dstnodata 255 "file01.ecw" "file03.tif"

file03.tiff 49396x46760

I did a try with -co PREDICTOR too.

Tools used here are FWTools 2.4.6 on Windows XP with an NTFS partition and GDAL 1.6.3 on Mandriva (with some personal packages to enable bigtiff's support) with an ext4 partition. Just for information, FME crashs on this conversion :)

(sorry for not having filled the version in this ticket but gdal 1.6.3 doesn't shows up in versions' list)

[warmerdam]

Hi,

To confirm you encounter the same problem on linux with 1.6.3 and with the FWTools on windows, right?

Have you tried warping to an uncompressed geotiff file?

[jrm]

Yes, the same issue on both system

I tried just now with fwtools without compression, the result is that the file get bigger faster

[Even Rouault]

gdalwarp had problems until recently when outputing to compressed geotiffs that could effictevly lead to a output file that is bigger than if you ask for uncompressed output.... Hopefully this has been improved lately in gdal 1.7.0dev

If you're in a position to build from source, you could try gdal 1.7.0beta2 to test the fix. Otherwise you could also try Tamas Szekerest' daily builds for Windows (the packages suffixed by -developement, not -stable ones) : ​http://vbkto.dyndns.org/sdk/ For the fix to be fully efficient, you should probably add -co TILED=YES

Otherwise if you stick with existing version, you can try to do a gdalwarp to an uncompressed file and then do a gdal_translate with compression option as suggesting in the FAQ : http://trac.osgeo.org/gdal/wiki/UserDocs/GdalWarp#GeoTIFFoutput-coCOMPRESSisbroken

"I tried just now with fwtools without compression, the result is that the file get bigger faster" : for uncompressed output, 49396x46760 = 2.3 GB (if you have a RGB dataset, it will be 2.3 * 3 = 6.9 GB) I really can't understand how this could get bigger than those figures.

[jrm]

I did try with gdalautotest-1.7b2, the output with COMPRESS=YES is correct this time.

For the uncompressed file I was talking about, I wasn't saying the output was larger than expected but that the rate of "grow" was faster. Sorry for the misunderstood.