gdal_translate fails on JPEG NITF image since band masks support

[Even Rouault]

Since the introduction of raster mask bands, "gdal_translate U_2001D.NTF tmp.tif" segfaults. U_2001D.NTF is the image quoted in ticket #1309 (the link to download it seems to be broken now. But I've a copy on my disk). I've checkout r11977 and it worked fine. It starts failing with r11978 where the JPEG driver has been updated with band mask support. (U_2001D.NTF is a JPEG compressed NITF image)

Here's Valgrind output on r12102 :

==4404== Invalid read of size 4
==4404==    at 0x42B60C6: GDALRasterBand::GetXSize() (gdalrasterband.cpp:2184)
==4404==    by 0x42A7A5C: CopyBandImageData(GDALRasterBand*, GDALRasterBand*, int (*)(double, char const*, void*), void*, double, double) (gdaldriver.cpp:166)
==4404==    by 0x42A7E56: GDALDriver::DefaultCopyMasks(GDALDataset*, GDALDataset*, int) (gdaldriver.cpp:255)
==4404==    by 0x42B0984: GDALPamDataset::CloneInfo(GDALDataset*, int) (gdalpamdataset.cpp:733)
==4404==    by 0x412E391: GTiffCreateCopy(char const*, GDALDataset*, int, char**, int (*)(double, char const*, void*), void*) (geotiff.cpp:4669)
==4404==    by 0x42A87B1: GDALDriver::CreateCopy(char const*, GDALDataset*, int, char**, int (*)(double, char const*, void*), void*) (gdaldriver.cpp:504)
==4404==    by 0x42A8977: GDALCreateCopy (gdaldriver.cpp:543)
==4404==    by 0x804B59C: ProxyMain(int, char**) (gdal_translate.cpp:575)
==4404==    by 0x804C108: main (gdal_translate.cpp:865)
==4404==  Address 0x1F is not stack'd, malloc'd or (recently) free'd
==4404==
==4404== Process terminating with default action of signal 11 (SIGSEGV)
==4404==  Access not within mapped region at address 0x1F
==4404==    at 0x42B60C6: GDALRasterBand::GetXSize() (gdalrasterband.cpp:2184)
==4404==    by 0x42A7A5C: CopyBandImageData(GDALRasterBand*, GDALRasterBand*, int (*)(double, char const*, void*), void*, double, double) (gdaldriver.cpp:166)
==4404==    by 0x42A7E56: GDALDriver::DefaultCopyMasks(GDALDataset*, GDALDataset*, int) (gdaldriver.cpp:255)
==4404==    by 0x42B0984: GDALPamDataset::CloneInfo(GDALDataset*, int) (gdalpamdataset.cpp:733)
==4404==    by 0x412E391: GTiffCreateCopy(char const*, GDALDataset*, int, char**, int (*)(double, char const*, void*), void*) (geotiff.cpp:4669)
==4404==    by 0x42A87B1: GDALDriver::CreateCopy(char const*, GDALDataset*, int, char**, int (*)(double, char const*, void*), void*) (gdaldriver.cpp:504)
==4404==    by 0x42A8977: GDALCreateCopy (gdaldriver.cpp:543)
==4404==    by 0x804B59C: ProxyMain(int, char**) (gdal_translate.cpp:575)
==4404==    by 0x804C108: main (gdal_translate.cpp:865)

[Even Rouault]

I've found out what is going wrong. In nitfdataset.cpp, we do the following :

poDS->SetBand( iBand+1, poDS->poJPEGDataset->GetRasterBand(iBand+1) );

The JPGRasterBand class has a JPGDataset *poGDS field that is set in its constructor and used by JPGRasterBand::IReadBlock for example. When doing poDS->setBand, the poDS field inherited from GDALRasterBand is updated to point onto the NITFDataset (poGDS still points onto the JPGDataSet*) In JPGRasterBand::GetMaskBand and JPGRasterBand::GetMaskFlags, we use this poDS inherited field and cast it to JPGDataset*, which is wrong here since it's a NITFDataset* ! Hence the segfault.

However, I'm not sure how it must be fixed. I let it to you Frank.

[warmerdam]

Even,

Thanks, I have a fix for this and will commit it when I have an extension for the test suite too.

[warmerdam]

Even,

Fix committed in r12105 and a gdalautotest check in r12104.

Thanks,