Crash when feeding data directly after creating a TIFF with JPEG compression and PHOTOMETRIC=YCBCR

[Even Rouault]

The following code snippet leads to a crash in GDALDatasetCopyWholeRaster or in the final GDALClose (when writing data to the disk indeed).

    GDALDatasetH hSrcDS = GDALCreate(GDALGetDriverByName("GTiff"), "src.tif",
                                     1000, 1000, 3, GDT_Byte, NULL); 
    GDALClose(hSrcDS);
    hSrcDS = GDALOpen("src.tif", GA_ReadOnly);

    char* options[] = {  "TILED=YES", "COMPRESS=JPEG", "PHOTOMETRIC=YCBCR", NULL };

    GDALDatasetH hDstDS = GDALCreate(GDALGetDriverByName("GTiff"), "dst.tif",
                                     GDALGetRasterXSize(hSrcDS),
                                     GDALGetRasterYSize(hSrcDS),
                                     GDALGetRasterCount(hSrcDS), GDT_Byte, options); 
  
    GDALDatasetCopyWholeRaster(hSrcDS, hDstDS, NULL, NULL, NULL);

    GDALClose(hDstDS);

The "PHOTOMETRIC=YCBCR" option is the reason for the crash. I suspect that something is wrong in libtiff, in the computation of the expected block size due to the resampling factors.

A workaround is to close hDstDS and reopen it in update mode (that's basically what is done in the CreateCopy of the GTiff driver, so we don't see this crash), but it is inconvenient for applications using GDAL API, and not wanting to use libtiff directly, as there is no way to set the jpeg quality tag for example.

[Even Rouault]

A full program to reproduce the problem

[Even Rouault]

OK, fixed in r15677 and test added in r15678. Backported in branches/1.5 in r15679

The bug fix is in 2 pieces :

• first one : set TIFFTAG_JPEGCOLORMODE=JPEGCOLORMODE_RGB when creating a TIFF with COMPRESS=JPEG and PHOTOMETRIC=YCBCR in GTiffDataset::Create()
• second one : similar to what is done in #2642 : save and reset TIFFTAG_JPEGCOLORMODE in Crystalize()