Opened 16 years ago

Closed 16 years ago

#1948 closed defect (fixed)

Update of LZW w/predictor compressed TIFF files leads to corrupted file

Reported by: Even Rouault Owned by: warmerdam
Priority: normal Milestone: 1.4.4
Component: GDAL_Raster Version: svn-trunk
Severity: normal Keywords: lzw libtiff gtiff predictor
Cc:

Description

GDAL utilities that tries to update some LZW compressed TIFF gives wrong results. For example, nearblack on http://veimages.gsfc.nasa.gov//2433/land_shallow_topo_2048.tif will give an image looking like a 'TV without signal' (quoting Mateusz) (This bug report primarly comes from #1936)

However, this is not reproductible on all LZW compressed TIFF. If you do :

gdal_translate -co "COMPRESS=LZW" land_shallow_topo_2048.tif tmp.tif
apps/nearblack tmp.tif

tmp.tif looks good.

But if you do :

gdal_translate -co "COMPRESS=LZW" -co "PREDICTOR=2" land_shallow_topo_2048.tif tmp.tif
apps/nearblack tmp.tif

the result is bad. (In that case, the translated tmp.tif has roughly the same size as land_shallow_topo_2048.tif since land_shallow_topo_2048.tif has PREDICTOR=2 tag)

But tmp.tif after translating is sigificantly larger than land_shallow_topo_2048.tif

(This is maybe related to #1758 as well.)

Change History (9)

comment:1 by Even Rouault, 16 years ago

Component: defaultGDAL_Raster
Keywords: lzw tiff added
Version: unspecifiedsvn-trunk

comment:2 by Even Rouault, 16 years ago

-co "PREDICTOR=2" seems to be the cause.

Similar problem can also be reproduced with http://dl.maptools.org/dl/gdal/data/nitf/cadrg/001zc013.on1

gdal_translate -co "COMPRESS=LZW" -co "PREDICTOR=2" 001zc013.on1 001zc013.tif
nearblack 001zc013.tif

gives wrong result, whereas samething without PREDICTOR=2 works fine.

comment:3 by Even Rouault, 16 years ago

The same applies for COMPRESS=DEFLATE and PREDICTOR=2

comment:4 by warmerdam, 16 years ago

Keywords: libtiff added
Status: newassigned

I have established that the libtiff predictor state is not being initialize and used for encoding when the same session uses decoding with LZW (and possibly other compression types).

Still digging in libtiff to see what the fix will be.

comment:5 by warmerdam, 16 years ago

Summary: Update of LZW compressed TIFF files leads to corrupted fileUpdate of LZW w/predictor compressed TIFF files leads to corrupted file

I have committed fixes to libtiff CVS head (libtiff4) to correct this. Modified tif_predict.{c,h}, as well as tif_lzw.c and tif_zip.c. Changes downstreamed into GDAL trunk as r12608.

I have also confirmed this bug occurs in 1.4 branch with libtiff3.9. I'd like to fix it there too.

comment:6 by warmerdam, 16 years ago

I have add modified autotest/gcore/tiff_write.py (test 7) to use PREDICTOR=2 to verify this fix is in place in trunk (r12609).

comment:7 by warmerdam, 16 years ago

Created #1965, and somewhat related bug related to predictors.

comment:8 by warmerdam, 16 years ago

The tif_write_7 test was still failing. It turns out that the pcode function was getting mixed up between encode and decode cases. Fixed in upstream libtiff by splitting pcode into encodepcode and decodepcode functions similarly to how the previous change was made. Test script now passing.

Still need to port into libtiff 3.9, and pull the fix downstream into GDAL.

comment:9 by warmerdam, 16 years ago

Keywords: gtiff predictor added; tiff removed
Milestone: 1.4.4
Resolution: fixed
Status: assignedclosed

I have backported this into libtiff 3.9 branch, and imported that branch into GDAL's 1.4 branch. I believe our predictor problems are corrected now.

Note: See TracTickets for help on using tickets.