gdal_translate failure when compiled with gcc 4.6.x

[bledoux]

Hello,

We have a problem with gdal translate and the following command line that is supposed to build a JPG preview out of a list of JP2000 tiles referenced by a VRT file. The VRT file is provided as an attachment but the raster data(over 500mb of JP2 tiles ) is too large. We believe that fake jp2 images could be used to reproduce the issue.

Command Line:

gdal_translate jp2_tiles_65lDbU.vrt PREVIEW_SPOT6_MS_20100616170800_ORT_ASTPROD_20110412192656325_1_1.JPG -of JPEG -ot Byte -b 1 -b 2 -b 3 -scale 0 4095 0 255 -outsize 1000 725 -co QUALITY=90 --config JP2KAK_THREADS 0

Result:

Input file size is 15443, 11201 0ERROR 5: Illegal nBlockXOff value (-11184810) in GDALRasterBand::TryGetLockedBlockRef?() ERROR 5: Illegal nBlockXOff value (-11184810) in GDALRasterBand::GetLockedBlockRef?() ERROR 5: Illegal nBlockXOff value (-1048576) in GDALRasterBand::TryGetLockedBlockRef?() ERROR 5: Illegal nBlockXOff value (-1048576) in GDALRasterBand::GetLockedBlockRef?()

Analysis

We have made further tests, using the same command line, but with different gcc and gdal versions (see the following table) and it appears that there is an incompatibility between gdal 1.8.x and gcc 4.6.x :

gcc 4.6.1 + gdal 1.8.1 + geos 3.3.1 Fail
gcc 4.6.1 + gdal 1.8.0 + geos 3.3.1 Fail
gcc 4.6.1 + gdal 1.8.0 + geos 3.1.1 Fail
gcc 4.1.1 + gdal 1.8.0 + geos 3.1.1 OK
gcc 4.1.1 + gdal 1.8.0 + geos 3.3.1 OK
gcc 4.1.1 + gdal 1.8.1 + geos 3.3.1 OK
gcc 4.6.1 + gdal 1.9.0 + geos 3.3.1 Fail
gcc 4.6.2 + gdal 1.9.0 + geos 3.3.1 Fail
gcc 4.5.3 + gdal 1.9.0 + geos 3.3.1 OK

[rouault]

I have no kakadu available, so I can only give some methodological advice to help you track down the root of the issue.

You could try to gdal_translate each tile individually in geotiff. Transform your VRT as a VRT of those GeoTIFF tiles, and test if you still observe the issue when doing the gdal_translate on the whole VRT.

Alternatively, use alternate JPEG2000 readers, like JP2ECW (from ECW SDK) or JP2MrSID (from MrSID SDK) and see if it makes a difference.

--> This should help knowing if the problem comes specifically from Kakadu or from GDAL Raster core.

You didn't mention if you use kakadu binaries compiled with the same compiler version, or if you recompiled them with the compiler used with GDAL. Perhaps something to test.

You could also try --enable-debug builds of GDAL to check if it comes from an optimization issue (make sure no -O2 comes in gcc/g++ command line. You can alternatively hand-edit GDALmake.opt to remove -O2 occurrences), make clean, make and retry.

With a debug-enabled build, running gdal_translate under valgrind could possible reveal interesting issues. Alternatively, you could run gdal_translate under gdb/ddd and attach a breakpoint on CPLError() and try to understand how negative values might have come to nBlockXOff.

Good luck !

[bledoux]

To answer rouault: All libraries (incl. kakadu) used are compiled using the same gcc version (namely gcc 4.6.1/2) .

We also made the test with TIFF tiles and the problem doesn't occur anymore, the generation of the preview succeeds.

That beeing said, it doesn't necessarily mean that the problem lies in the kakadu library. We use the it in another context, in order to generate the JP2000 tiles, and have no such problems.

We'll do some more testing/debugging and hopefully come with an explanation/solution.

[bledoux]

Supplementary information:

We made further tests at different gcc optimization levels and using other versions of gcc and kakadu.
The table below sums up the results:

GCC , KDU, -O, Result
4.5.3, 6.3/6.4, none/1/2, OK
4.6.x, 6.3/6.4, none, OK
4.6.x, 6.3/6.4, 1/2, Failed

Conclusions :

-One workaround is to disable the optimizations when buidling Kakadu with gcc 4.6.1.
-The other would be to use gcc 4.5.3 but we can't for other reasons (openmp implementation issues...).
-We will log a bug to the kakadu library distributor.

[rouault]

ok, so closing as not a (GDAL) bug