[PATCH - libtiff] JPEG compression in TIF doesn't work

[whimbrel]

When an image height is not a multiple of the strip/block size, compressing a TIF with JPEG compression in gdal_translate returns the error message:

Warning 1: JPEGLib:Application transferred too many scanlines

And the last rows of the image are not processed (given value 0).

For example I have an 8-bit aerial photo as GeoTiff, packbits compressed, which measuers 9728c x 13963r. I want to convert it with gdal_translate to JPEG-compressed with 50% JPEG quality.

I use: gdal_translate -ot Byte -of GTiff -co "COMPRESS=JPEG" -co "JPEG_QUALITY=50" allagash-lake.tif test_jpeg50_tif.tif

It goes through most of the image, finishes with this error message: Warning 1: JPEGLib:Application transferred too many scanlines

The resulting image is processed correctly for the first 13952 lines, but the last 11 are value 0. 13952 is a multiple of 16. If I had 5 more lines in my image it would probably work.

Michael Smith Maine Office of GIS michael.smith(at)maine.gov

[Even Rouault]

Michael, as a workaround, you can try to generate tiled GeoTIFF by adding -co "TILED=YES". This may also be an improvement as tiling may be more appropriate depending on your use cases.

Frank, I've tried successfully the following patch to libtiff :

Index: frmts/gtiff/libtiff/tif_jpeg.c
===================================================================
--- frmts/gtiff/libtiff/tif_jpeg.c	(révision 15985)
+++ frmts/gtiff/libtiff/tif_jpeg.c	(copie de travail)
@@ -1713,6 +1713,7 @@
 JPEGEncode(TIFF* tif, uint8* buf, tmsize_t cc, uint16 s)
 {
 	JPEGState *sp = JState(tif);
+	TIFFDirectory *td = &tif->tif_dir;
 	tmsize_t nrows;
 	JSAMPROW bufptr[1];
 
@@ -1723,7 +1724,7 @@
 	if (cc % sp->bytesperline)
 		TIFFWarningExt(tif->tif_clientdata, tif->tif_name, "fractional scanline discarded");
 
-	while (nrows-- > 0) {
+	while (nrows-- > 0 && (isTiled(tif) || tif->tif_row < td->td_imagelength)) {
 		bufptr[0] = (JSAMPROW) buf;
 		if (TIFFjpeg_write_scanlines(sp, bufptr, 1) != 1)
 			return (0);

This will make the following test work :

import gdal

ds = gdal.GetDriverByName('GTiff').Create('test_strip_jpeg.tif', 4, 4000, 1, options = ['COMPRESS=JPEG'] )
ds.GetRasterBand(1).Fill(255)
ds = None

ds2 = gdal.GetDriverByName('GTiff').Create('test_strip.tif', 4, 4000, 1 )
ds2.GetRasterBand(1).Fill(255)
ds2 = None

ds = gdal.Open('test_strip_jpeg.tif')
ds2 = gdal.Open('test_strip.tif')
if ds.GetRasterBand(1).Checksum() != ds2.GetRasterBand(1).Checksum():
    print 'error'

[warmerdam]

This has been corrected in libtiff per ​http://bugzilla.maptools.org/show_bug.cgi?id=1981

The patch is not exactly the same, but similar.

I'm still wondering whether GDAL ought to be trimming the data written to the last strip itself. It seems like the libtiff tools generally do this, though jpeg was the only compressor I discovered that errored out.

I'm keeping this ticket open to consider special logic for last partial strips (to go in GTiffDataset::WriteEncodedStrip().

[Even Rouault]

Apparently, the logic has been added in trunk in r15988, and in branches/1.5 in r15989. Test added in r15996