libtiff PredictorEncodeTile alters passed buffer

[warmerdam]

PredictorEncodeTile?() in tif_predict.c alters the passed in buffer as it applies the predictor transformation. But the passed in buffer is assumed by code such as GTiffRasterBand::IWriteBlock() to *not* be altered by the call to TIFFWriteEncodedTile().

This function needs to be altered to do the predictor transformation into a temporary buffer instead of altering the original buffer.

This issue presumably exists in all libtiff versions, and results in corrupted GDAL block caches in some cases involving reading and writing to LZW or Deflate compressed TIFF files with an active predictor.

Currently this is causing tif_write_7 to fail in the autotest suite.

[warmerdam]

Corrected upstream in libtiff head (libtiff 4.0) by creating a temporary working buffer to operate in in PredictorEncodeTile?().

I will need to refresh GDAL's libtiff version, and likely back port this change into libtiff 3.9.

[rouault]

I'm not sure if it relates with this ticket, but I observe the following warning with Valgrind on tiff_write_6 (r12824 build with --libtiff=internal) :

==14115== Conditional jump or move depends on uninitialised value(s)
==14115==    at 0x54D7CD2: (within /usr/lib/libz.so.1.2.3.3)
==14115==    by 0x54D8E5F: deflate (in /usr/lib/libz.so.1.2.3.3)
==14115==    by 0x484D05F: ZIPPostEncode (tif_zip.c:296)
==14115==    by 0x484BA95: TIFFWriteEncodedTile (tif_write.c:399)
==14115==    by 0x46EA02B: GTiffRasterBand::IWriteBlock(int, int, void*) (geotiff.cpp:596)
==14115==    by 0x4893E1A: GDALRasterBlock::Write() (gdalrasterblock.cpp:304)
==14115==    by 0x4891DE2: GDALRasterBand::FlushBlock(int, int) (gdalrasterband.cpp:932)
==14115==    by 0x4891E7F: GDALRasterBand::FlushCache() (gdalrasterband.cpp:773)
==14115==    by 0x487AF13: GDALDataset::FlushCache() (gdaldataset.cpp:180)
==14115==    by 0x488A794: GDALPamDataset::FlushCache() (gdalpamdataset.cpp:155)
==14115==    by 0x46E7B72: GTiffDataset::FlushCache() (geotiff.cpp:2033)
==14115==    by 0x487BDEA: GDALFlushCache (gdaldataset.cpp:197)
==14115==    by 0x4529DFE: _wrap_Dataset_FlushCache (gdal_wrap.cpp:3186)
==14115==    by 0x805C9E6: PyObject_Call (in /usr/bin/python2.5)
==14115==    by 0x80C709E: PyEval_EvalFrameEx (in /usr/bin/python2.5)
==14115==    by 0x80CA114: PyEval_EvalCodeEx (in /usr/bin/python2.5)
==14115==    by 0x80C85D8: PyEval_EvalFrameEx (in /usr/bin/python2.5)
==14115==    by 0x80C9314: PyEval_EvalFrameEx (in /usr/bin/python2.5)
==14115==    by 0x80C9314: PyEval_EvalFrameEx (in /usr/bin/python2.5)
==14115==    by 0x80CA114: PyEval_EvalCodeEx (in /usr/bin/python2.5)

[warmerdam]

Predictor changes ported into libtiff 3.9 branch, and libtiff 3.9 branch imported into gdal/frmts/gtiff/libtiff.

PS. I don't think the valgrind report is related to this issue.