gdalinfo crashes against orbview nitf imagery

[abarton]

Gdalinfo.exe version 1.6.3-1.6.1 crashes when ran against some OrbView?-5 imagery. While running it continues to be cpu intensive and use more memory until it's failure. I've found no relation between filesize and memory usage; my test files are 2.7GB (1 band) and 1.2GB (4 band) while generally gdalinfo uses ~ 1.75GB before failing.

I've found that gdalinfo.exe version 1.6.0dev, FWTools 2.1.0 is able to read all samples that fail in subsequent versions. Attached is output from the working version of gdalinfo for both test cases. Associated sample data, orbview-5_4x8U.ntf and orbview-5_8U.ntf, can be found at ftp.i3.com using the fwarmerdam login.

[warmerdam]

Yusuf / Austin,

I have downloaded orbview-5_4x8U.ntf (1.2GB) to my external linux server, and run gdalinfo 1.6, and trunk against it with no problem. I have downloaded the first 3MB and last 3MB to home, and reconstituted something approximating the original file (with zeros in the middle) and tested that with gdal16 from osgeo4w, and FWTools 2.4.2 and encountered no problems with gdalinfo.

I'm afraid it is not practical for me to move such a large file home over my satellite connection, and I have been unsuccessful reproducing a problem on linux. I'm not sure how to pursue this further. One possibility would be for you to send me the image(s) on DVD.

Looking at the gdalinfo reports Austin attached, I see reference to an .aux file, but that does not seem to be on the ftp site. Could you supply those? If you mv the .aux files away from the .ntf file, does gdalinfo work then?

[abarton]

I incorrectly cited orbview-5_4x8U.ntf as a problem file. I've confirmed that the other file, orbview-5_8U.ntf, generates the issue as described in the bug. Could you please test against that.

[warmerdam]

I have reproduced the problem with older FWTools 2.4.2 and with OSGeo4W's 1.6.3-2 package if the ecw plugin is available but the jp2kak one is not.

The problem is handling of large (>2GB) subfile stream in the nitf and ecw code in 1.6 branch. This issue has already been corrected in trunk and 1.7 via another ticket and use of /vsisubfile/.

I have applied a minimal fix in 1.6 branch (r18747) and am repackaging the OSGeo4W gdal16 and gdal16-ecw packages with the fix.

The latests FWTools already fixes the problem.

[warmerdam]

new gdal16-1.6.3-3 and gdal16-ecw-1.6.3-1 packages produced with the bug fix.

[abarton]

The bug is reproducible using 1.6.3-3.

[warmerdam]

I believe things did work once gdal16-ecw was upgraded as well.