[raster] Segfault when doing raster2pgsql

[thesteve0]

When trying to run raster2pgsql on Fedora 19 to process a geotiff I get a segfault

psql (PostgreSQL) 9.2.4 GDAL 1.9.2, released 2012/10/08

The file I am using is over 100 megs so I don't think I should attach it to the ticket.

(gdb) run -s 3309 -t 100x100 -I -M -l 4 IR_R_G.tif public.erdastest > gdb.sql Starting program: /usr/bin/raster2pgsql -s 3309 -t 100x100 -I -M -l 4 IR_R_G.tif public.erdastest > gdb.sql [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Processing 1/1: IR_R_G.tif

Program received signal SIGSEGV, Segmentation fault. 0x0000003d7667c44d in malloc_consolidate () from /usr/lib64/libc.so.6 (gdb) bt #0 0x0000003d7667c44d in malloc_consolidate () from /usr/lib64/libc.so.6 #1 0x0000003d7667cfd6 in _int_free () from /usr/lib64/libc.so.6 #2 0x0000003a0c418278 in gtCSVDeaccess () from /usr/lib64/libgeotiff.so.1.2 #3 0x0000003a0c4182c9 in gtCSVDeaccess () from /usr/lib64/libgeotiff.so.1.2 #4 0x0000003a0934a8c2 in GDALDriver::~GDALDriver() ()

from /usr/lib64/libgdal.so.1

#5 0x0000003a0934a8f9 in GDALDriver::~GDALDriver() ()

from /usr/lib64/libgdal.so.1

#6 0x0000003a0934cac0 in GDALDriverManager::~GDALDriverManager() ()

from /usr/lib64/libgdal.so.1

#7 0x0000003a0934caf9 in GDALDriverManager::~GDALDriverManager() ()

from /usr/lib64/libgdal.so.1

#8 0x0000003a090e37ce in GDALDestroy() () from /usr/lib64/libgdal.so.1 #9 0x0000003d75e0fb6a in _dl_fini () from /lib64/ld-linux-x86-64.so.2 #10 0x0000003d766392a9 in run_exit_handlers () from /usr/lib64/libc.so.6 #11 0x0000003d766392f5 in exit () from /usr/lib64/libc.so.6 #12 0x0000003d76621b7c in libc_start_main () from /usr/lib64/libc.so.6 #13 0x00000000004053a5 in _start ()

[thesteve0]

Missing separate debuginfos, use: debuginfo-install postgis-2.0.3-1.fc19.x86_64
(gdb) -s 3309 -t 100x100 -I -M -l 4 IR_R_G.tif  public.erdastest > gdb.sql
Undefined command: "-s".  Try "help".
(gdb) run -s 3309 -t 100x100 -I -M -l 4 IR_R_G.tif  public.erdastest > gdb.sql
Starting program: /usr/bin/raster2pgsql -s 3309 -t 100x100 -I -M -l 4 IR_R_G.tif  public.erdastest > gdb.sql
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Processing 1/1: IR_R_G.tif

Program received signal SIGSEGV, Segmentation fault.
0x0000003d7667c44d in malloc_consolidate () from /usr/lib64/libc.so.6
(gdb) bt
#0  0x0000003d7667c44d in malloc_consolidate () from /usr/lib64/libc.so.6
#1  0x0000003d7667cfd6 in _int_free () from /usr/lib64/libc.so.6
#2  0x0000003a0c418278 in gtCSVDeaccess () from /usr/lib64/libgeotiff.so.1.2
#3  0x0000003a0c4182c9 in gtCSVDeaccess () from /usr/lib64/libgeotiff.so.1.2
#4  0x0000003a0934a8c2 in GDALDriver::~GDALDriver() ()
   from /usr/lib64/libgdal.so.1
#5  0x0000003a0934a8f9 in GDALDriver::~GDALDriver() ()
   from /usr/lib64/libgdal.so.1
#6  0x0000003a0934cac0 in GDALDriverManager::~GDALDriverManager() ()
   from /usr/lib64/libgdal.so.1
#7  0x0000003a0934caf9 in GDALDriverManager::~GDALDriverManager() ()
   from /usr/lib64/libgdal.so.1
#8  0x0000003a090e37ce in GDALDestroy() () from /usr/lib64/libgdal.so.1
#9  0x0000003d75e0fb6a in _dl_fini () from /lib64/ld-linux-x86-64.so.2
#10 0x0000003d766392a9 in __run_exit_handlers () from /usr/lib64/libc.so.6
#11 0x0000003d766392f5 in exit () from /usr/lib64/libc.so.6
#12 0x0000003d76621b7c in __libc_start_main () from /usr/lib64/libc.so.6
#13 0x00000000004053a5 in _start ()

[robe]

Steve, You have a download link to the file?

[Bborie Park]

From the backtrace, the problem appears to be in GDAL or the external GeoTIFF library. Is it possible to try a GDAL built with its internal GeoTIFF library?

[TheSteve0]

Here is a link to the file ​https://dl.dropboxusercontent.com/u/1301514/IR_R_G.tif

I can't. I am just trying to use the packaged version that comes with Fedora 19. Let me know how you would like to proceed. Thanks Steve

[Bborie Park]

Just tested PostGIS 2.0 and 2.1 using GDAL trunk built with internal GeoTIFF support. So, the problem definitely is not in PostGIS.

[TheSteve0]

Ok so then it is a postgis packaging/GDAL packaging issue on Fedora right? If so how do you think I should communicate the problem to them?

[Bborie Park]

I can't say that it is a fedora packaging issue. Can you try running gdalinfo on the raster file and see if it has issues?

[thesteve0]

Works fine

[spousty@localhost tmp]$ gdalinfo IR_R_G.tif 
Driver: GTiff/GeoTIFF
Files: IR_R_G.tif
Size is 7794, 7260
Coordinate System is:
PROJCS["unnamed",
    GEOGCS["NAD83",
        DATUM["North_American_Datum_1983",
            SPHEROID["GRS 1980",6378137,298.2572221010002,
                AUTHORITY["EPSG","7019"]],
            AUTHORITY["EPSG","6269"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4269"]],
    PROJECTION["Albers_Conic_Equal_Area"],
    PARAMETER["standard_parallel_1",34],
    PARAMETER["standard_parallel_2",40.5],
    PARAMETER["latitude_of_center",0],
    PARAMETER["longitude_of_center",-120],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",-4000000],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]]]
Origin = (-311711.607355843007099,51188.279698056103371)
Pixel Size = (30.000000000000000,-30.000000000000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  ( -311711.607,   51188.280) (123d34'31.75"W, 38d25'26.25"N)
Lower Left  ( -311711.607, -166611.720) (123d29' 1.42"W, 36d27'57.42"N)
Upper Right (  -77891.607,   51188.280) (120d53'37.87"W, 38d28'24.73"N)
Lower Right (  -77891.607, -166611.720) (120d52'15.22"W, 36d30'51.36"N)
Center      ( -194801.607,  -57711.720) (122d12'21.77"W, 37d28'36.59"N)
Band 1 Block=7794x1 Type=Byte, ColorInterp=Red
Band 2 Block=7794x1 Type=Byte, ColorInterp=Green
Band 3 Block=7794x1 Type=Byte, ColorInterp=Blue

[Bborie Park]

How about

gdalinfo -stats IR_R_G.tif

[thesteve0]

Works great as well. It is something about the -l that is making things go wonky

[Bborie Park]

Ah. That would explain it. -l makes use of a VRT on top of the source raster to generate overviews. Can you create a VRT on that raster with gdalbuildvrt and test gdalinfo on that generated VRT?

[thesteve0]

Can you explain (or just paste the parameters) on how you would use it?

[Bborie Park]

I've found that some raster format drivers have issues with be wrapped with a VRT. These issues haven't happened consistently but they have always been related to VRTs.

Try something like the following…

gdalbuildvrt -tr 120 120 test.vrt IR_R_G.tif

gdalinfo -stats test.vrt

Based upon how that gdalinfo returns, I'll do some testing to see if raster2pgsql's aggressive use of GDALClose() is conflicting with VRT's handling of GDAL datasets.

[strk]

thesteve0: did you ever handle to try what Bborie suggested ?

[Bborie Park]

Closing due lack of activity and that the problem was diagnosed to a driver/VRT interaction issue.