gdalbuildvrt trunk crash on MacOSX

[jpalmer]

gdalbuildvrt -resolution highest test.vrt 92LBZ-92L3G.tif
0...10...20...30...40...50...60...70...80...90...100 - done.
TIFFReadDirectory: Warning, Unknown field with tag 42113 (0xa481) encountered.
Segmentation fault: 11

Using this example file with a brew built version of gdal trunk on MacOSX 10.10 ​https://dl.dropboxusercontent.com/u/30623980/92LBZ-92L3G.tif

back trace returns:

(lldb) file /usr/local/bin/gdalbuildvrt
Current executable set to '/usr/local/bin/gdalbuildvrt' (x86_64).
(lldb) r -resolution highest test.vrt 92LBZ-92L3G.tif
Process 73978 launched: '/usr/local/bin/gdalbuildvrt' (x86_64)
0...10...20...30...40...50...60...70...80...90...100 - done.
TIFFReadDirectory: Warning, Unknown field with tag 42113 (0xa481) encountered.
Process 73978 stopped
* thread #1: tid = 0x1390e8, 0x000000010510b418 libtiff.5.dylib`_TIFFVGetField + 1212, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa00)
    frame #0: 0x000000010510b418 libtiff.5.dylib`_TIFFVGetField + 1212
libtiff.5.dylib`_TIFFVGetField:
->  0x10510b418 <+1212>: movq   %rax, (%rcx)
    0x10510b41b <+1215>: movl   $0x1, %r12d
    0x10510b421 <+1221>: movl   %r12d, %eax
    0x10510b424 <+1224>: addq   $0x8, %rsp
(lldb) bt
* thread #1: tid = 0x1390e8, 0x000000010510b418 libtiff.5.dylib`_TIFFVGetField + 1212, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa00)
  * frame #0: 0x000000010510b418 libtiff.5.dylib`_TIFFVGetField + 1212
    frame #1: 0x00000001045a9702 libtiff.5.dylib`TIFFVGetField + 178
    frame #2: 0x00000001045a9607 libtiff.5.dylib`TIFFGetField + 359
    frame #3: 0x00000001000cd945 libgdal.20.dylib`GTiffDataset::OpenOffset(tiff*, GTiffDataset**, unsigned long, int, GDALAccess, int, int, char**) + 1637
    frame #4: 0x00000001000d2b7a libgdal.20.dylib`GTiffDataset::Open(GDALOpenInfo*) + 3804
    frame #5: 0x0000000100290e38 libgdal.20.dylib`GDALOpenEx + 967
    frame #6: 0x000000010000c2de gdalbuildvrt`VRTBuilder::Build(int (*)(double, char const*, void*), void*) + 798
    frame #7: 0x000000010000d089 gdalbuildvrt`main + 2778
    frame #8: 0x00007fff8fec85c9 libdyld.dylib`start + 1

configure output for build is here:

​https://gist.github.com/palmerj/df99eb8e65d316b66efb

[Even Rouault]

Jeremy, several questions as I don't have access to a Mac :

• is this crash specific of GDAL 2.0/trunk, or can also it be reproduced with 1.11 ?
• is this crash specific of this particular geotiff file ?
• could it be possible to have a GDAL build with debug symbols (./configure --enable-debug) so as to have the line number in geotiff.cpp where the crash occurs ?
• what is the libtiff version used ?
• could the GDAL build be done with ./configure --with-libtiff=internal --rename_internal_libgeotiff_symbols ?
• does it crash just with gdalbuildvrt ? I'd suspect gdalinfo to crash too, no ?

[jpalmer]

• Using the installed frameworks from KyngChaos and it works fine. This is version 1.11
• No I can get to cash with all TIFFs I've tested so far
• yes gdalinfo crashes as well
• version of libtiff was:

brew info libtiff libtiff: stable 4.0.3 (bottled) ​http://www.remotesensing.org/libtiff/ /usr/local/Cellar/libtiff/4.0.3 (254 files, 3.8M) *

Poured from bottle

From: ​https://github.com/Homebrew/homebrew/blob/master/Library/Formula/libtiff.rb ==> Dependencies Required: jpeg ✔ ==> Options --c++11

Build using C++11 mode

--universal

Build a universal binary

I built with --enabled-debug, --with-libtiff=internal --with-rename_internal_libgeotiff_symbols=yes and then the crash went away. So issue with external libtiff?

Thanks Jeremy

[Even Rouault]

Just tried on Linux with external libtiff 4.0.3 (I tried also enabling c++11 just in case) and works well. Could you try tiffdump (installed with libtiff) and listgeo (installed with libgeotiff) utilities on the TIFF files that crash with GDAL ?

[jpalmer]

tiffdump seems to work:

tiffdump 92LBZ-92L3G.tif
92LBZ-92L3G.tif:
Magic: 0x4949 <little-endian> Version: 0x2a <ClassicTIFF>
Directory 0: offset 8 (0x8) next 0 (0)
ImageWidth (256) SHORT (3) 1<8192>
ImageLength (257) SHORT (3) 1<2777>
BitsPerSample (258) SHORT (3) 1<32>
Compression (259) SHORT (3) 1<5>
Photometric (262) SHORT (3) 1<1>
StripOffsets (273) LONG (4) 2777<22611 23220 23829 24438 25060 25689 26323 26960 27600 28242 28887 29534 30183 30833 31485 32139 32794 33451 34110 34770 35431 36092 36756 37420 ...>
SamplesPerPixel (277) SHORT (3) 1<1>
RowsPerStrip (278) SHORT (3) 1<1>
StripByteCounts (279) LONG (4) 2777<609 609 609 622 629 634 637 640 642 645 647 649 650 652 654 655 657 659 660 661 661 664 664 665 ...>
PlanarConfig (284) SHORT (3) 1<1>
Predictor (317) SHORT (3) 1<1>
SampleFormat (339) SHORT (3) 1<3>
33550 (0x830e) DOUBLE (12) 3<8 8 0>
33922 (0x8482) DOUBLE (12) 6<0 0 0 1.97263e+07 -4.43424e+06 0>
34735 (0x87af) SHORT (3) 32<1 1 0 7 1024 0 1 1 1025 0 1 2 1026 34737 25 0 2049 34737 7 25 2054 0 1 9102 ...>
34737 (0x87b1) ASCII (2) 33<WGS 84 / Pseudo-Mercator ...>
42113 (0xa481) ASCII (2) 7<-32767\0>

[jpalmer]

List list geo aso works:

listgeo 92LBZ-92L3G.tif
TIFFReadDirectory: Warning, Unknown field with tag 42113 (0xa481) encountered.
Geotiff_Information:
   Version: 1
   Key_Revision: 1.0
   Tagged_Information:
      ModelTiepointTag (2,3):
         0                 0                 0                
         19726340          -4434236.30242    0                
      ModelPixelScaleTag (1,3):
         8                 8                 0                
      End_Of_Tags.
   Keyed_Information:
      GTModelTypeGeoKey (Short,1): ModelTypeProjected
      GTRasterTypeGeoKey (Short,1): RasterPixelIsPoint
      GTCitationGeoKey (Ascii,25): "WGS 84 / Pseudo-Mercator"
      GeogCitationGeoKey (Ascii,7): "WGS 84"
      GeogAngularUnitsGeoKey (Short,1): Angular_Degree
      ProjectedCSTypeGeoKey (Short,1): Unknown-3857
      ProjLinearUnitsGeoKey (Short,1): Linear_Meter
      End_Of_Keys.
   End_Of_Geotiff.

PCS = 3857 (WGS 84 / Pseudo-Mercator)
Projection = 3856 (Popular Visualisation Pseudo-Mercator)
Projection Method: CT_Mercator
   ProjNatOriginLatGeoKey: 0.000000 (  0d 0' 0.00"N)
   ProjNatOriginLongGeoKey: 0.000000 (  0d 0' 0.00"E)
   ProjScaleAtNatOriginGeoKey: 1.000000
   ProjFalseEastingGeoKey: 0.000000 m
   ProjFalseNorthingGeoKey: 0.000000 m
GCS: 4326/WGS 84
Datum: 6326/World Geodetic System 1984
Ellipsoid: 7030/WGS 84 (6378137.00,6356752.31)
Prime Meridian: 8901/Greenwich (0.000000/  0d 0' 0.00"E)
Projection Linear Units: 9001/metre (1.000000m)

Corner Coordinates:
Upper Left    (19726340.000,-4434236.302)  (177d12'17.02"E, 37d 9' 0.18"S)
Lower Left    (19726340.000,-4456452.302)  (177d12'17.02"E, 37d18'34.67"S)
Upper Right   (19791876.000,-4434236.302)  (177d47'36.41"E, 37d 9' 0.18"S)
Lower Right   (19791876.000,-4456452.302)  (177d47'36.41"E, 37d18'34.67"S)
Center        (19759108.000,-4445344.302)  (177d29'56.71"E, 37d13'47.58"S)

[Even Rouault]

Jeremy, would you be able to apply the same brew build recipee, all things being equal (i.e. against external libtiff 4.0.3), except using GDAL 1.11 ? I'd like to know if it is a GDAL 2.0 regression before promoting RC1 to final.

[jpalmer]

Still crashing when I build with external lib:

--with-libtiff=#{HOMEBREW_PREFIX} --with-geotiff=#{HOMEBREW_PREFIX}, --with-rename-internal-libgeotiff-symbols=yes

Funny I built with debug but no symbols:

{{(lldb) r 92LBZ-92L3G.tif Process 82610 launched: '/usr/local/bin/gdalinfo' (x86_64) TIFFReadDirectory: Warning, Unknown field with tag 42113 (0xa481) encountered. Process 82610 stopped

• thread #1: tid = 0x22d2b5, 0x000000010528b418 libtiff.5.dylib`_TIFFVGetField + 1212, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa00)

frame #0: 0x000000010528b418 libtiff.5.dylib`_TIFFVGetField + 1212

libtiff.5.dylib`_TIFFVGetField: -> 0x10528b418 <+1212>: movq %rax, (%rcx)

0x10528b41b <+1215>: movl $0x1, %r12d 0x10528b421 <+1221>: movl %r12d, %eax 0x10528b424 <+1224>: addq $0x8, %rsp

(lldb) bt

• thread #1: tid = 0x22d2b5, 0x000000010528b418 libtiff.5.dylib`_TIFFVGetField + 1212, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa00)
  • frame #0: 0x000000010528b418 libtiff.5.dylib`_TIFFVGetField + 1212 frame #1: 0x0000000104729702 libtiff.5.dylib`TIFFVGetField + 178 frame #2: 0x0000000104729607 libtiff.5.dylib`TIFFGetField + 359 frame #3: 0x00000001000cfe09 libgdal.20.dylib`GTiffDataset::OpenOffset(tiff*, GTiffDataset, unsigned long, int, GDALAccess, int, int, char) + 1637 frame #4: 0x00000001000d50f1 libgdal.20.dylib`GTiffDataset::Open(GDALOpenInfo*) + 3911 frame #5: 0x0000000100296626 libgdal.20.dylib`GDALOpenEx + 999 frame #6: 0x000000010000ba43 gdalinfo`main + 1332 frame #7: 0x00007fff8fec85c9 libdyld.dylib`start + 1

}}

[Even Rouault]

Replying to jpalmer:

Still crashing when I build with external lib:

--with-libtiff=#{HOMEBREW_PREFIX} --with-geotiff=#{HOMEBREW_PREFIX}, --with-rename-internal-libgeotiff-symbols=yes

You mean with GDAL 2.0/trunk ? I'd note that --with-rename-internal-libgeotiff-symbols=yes has no effect for external libgeotiff

I'm wondering if the crash you get couldn't be related to possibly GDAL linking to another lib that would link against libtiff 3.X causing symbol clashes. I'm not sure which is the Mac equivalent of the Linux ldd tool to display the libraries linked with a shared object or binary, but that would be worth checking if GDAL isn't linking against 2 different versions of libtiff

[jpalmer]

Ah sorry yip my simple mistake.

otool -L /usr/local/bin/gdalinfo | grep -i tif
	/usr/local/lib/libgeotiff.2.dylib (compatibility version 4.0.0, current version 4.1.0)
	/Applications/Postgres.app/Contents/Versions/9.4/lib/libtiff.5.dylib (compatibility version 8.0.0, current version 8.0.0)

If I remove the postgres source the crash stops.

[Even Rouault]

I suppose the ticket can be closed ?

[jpalmer]

Yes.

[Even Rouault]

Milestone renamed