Opened 7 years ago

Last modified 7 years ago

#3365 closed defect

r.in.gdal: crash on large dataset — at Initial Version

Reported by: neteler Owned by: grass-dev@…
Priority: normal Milestone: 7.2.2
Component: Raster Version: svn-releasebranch72
Keywords: r.in.gdal Cc:
CPU: x86-64 Platform: Linux

Description

Trying to import a large VRT dataset, I get a crash:

GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-linux-gnu > r.in.gdal input=/mnt/kvm7-storage/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt output=loss_global_forest_cover_loss_2000_2015
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
Segmentation fault (core dumped)

Debugging:

GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-linux-gnu > gdb r.in.gdal
GNU gdb (GDB) Fedora 7.12.1-48.fc25
Copyright (C) 2017 Free Software Foundation, Inc.
[...]
Reading symbols from r.in.gdal...done.
(gdb) r input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt output=loss_global_forest_cover_loss_2000_2015
Starting program: /home/mundialis/software/grass72_svn_debugmode/dist.x86_64-pc-linux-gnu/bin/r.in.gdal input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt output=loss_global_forest_cover_loss_2000_2015
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.24-8.fc25.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff77a1ab9 in put_data (fd=0, null_buf=0x7fffffe9b020 "", cell=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0) at put_row.c:370
370		compressed_buf[0] = work_buf[0] = nbytes;



(gdb) bt full
#0  0x00007ffff77a1ab9 in put_data (fd=0, null_buf=0x7fffffe9b020 "", cell=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0) at put_row.c:370
        wk = 0x7fffff91cb31 ""
        nbytes = 1
        compressed_buf = 0x7fffff7bd220 <error: Cannot access memory at address 0x7fffff7bd220>
        total = 1440000
        fcb = 0x691080
        compressed = 1
        len = 4
        work_buf = 0x7fffff91cb30 "\001"
        wk = 0x7fffff91cb31 ""
        nwrite = 0
#1  0x00007ffff77a1f98 in put_raster_data (fd=0, null_buf=0x7fffffe9b020 "", rast=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0, map_type=0)
    at put_row.c:476
        fcb = 0x691080
#2  0x00007ffff77a2bdb in put_raster_row (fd=0, buf=0x7fffc62ac010, data_type=0, zeros_r_nulls=0) at put_row.c:699
        convert_and_write_FtypeOtype = {{0x0, 0x7ffff77a23d9 <convert_and_write_if>, 0x7ffff77a25f5 <convert_and_write_id>}, {
            0x7ffff77a280f <convert_and_write_fi>, 0x0, 0x7ffff77a26f3 <convert_and_write_fd>}, {0x7ffff77a2928 <convert_and_write_di>, 
            0x7ffff77a24d7 <convert_and_write_df>, 0x0}}
        fcb = 0x691080
        null_buf = 0x7fffffe9b020 ""
#3  0x00007ffff77a0f8b in Rast_put_row (fd=0, buf=0x7fffc62ac010, data_type=0) at put_row.c:67
No locals.
#4  0x0000000000406e4d in ImportBand (hBand=0x7dbcb0, output=0x627dc0 "loss_global_forest_cover_loss_2000_2015", group_ref=0x0) at main.c:1141
        data_type = 0
        eGDT = GDT_Int32
        eRawGDT = GDT_Byte
        row = 1
        nrows = 560000
        ncols = 1440000
        complex = 0
        cf = 0
        cfR = 32767
        cfI = -145625496
        bNoDataEnabled = 0
        indx = 32767
        cell = 0x7fffc62ac010
        cellReal = 0x77
        cellImg = 0x7fffffffac40
        bufComplex = 0x7fffffffab40
        dfNoData = -10000
        outputReal = "0\341\377\367\377\177\000\000\260\254\377\377\377\177\000\000\071\031@\000\000\000\000\000\243\366\251\242\000\000\000\000\377\377\377\377", '\000' <repeats 12 times>, "H\017\064\366\377\177\000\000\000\240\374\367\377\177\000\000\000\000\000\000\000\000\360?", '\000' <repeats 14 times>, "\360?", '\000' <repeats 30 times>, "\360?\000\000\000\000\000\000\000\000\220\062@\000\000\000\000\000\314\345w\367\377\177\000\000P\255\377\377\377\177\000\000\350\262`\000\000\000\000\000\220\062@\000\000\000\000\000\320\332\377\377\377\177", '\000' <repeats 18 times>, "\360\331\377\377\377\177\000\000"...
        outputImg = '\000' <repeats 32 times>, "\377\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\377\377\377\377\000\377\377\377", '\000' <repeats 22 times>, "\360?", '\000' <repeats 14 times>, "\360?", '\000' <repeats 14 times>, "\360?\000\000\000\000\000\000\000\000\210\345w\367\377\177", '\000' <repeats 26 times>, "\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\200\256\377\377\377\177\000\000\240\224z\367\377\177\000\000\000\273}", '\000' <repeats 13 times>...
        nullFlags = 0x0
        history = {fields = {0x7ffff7fca000 "", 0x7fffffffabe8 "", 0x7fffffffabe4 "", 0x7ffff7ff7b78 "\350\343\377\367\377\177", 
            0x7fffffffaba0 "\300\254\377\377\377\177", 0x401939 "GDALGetRasterBand", 0x400be8 "\261\a", 0x7fffffffabe8 ""}, nlines = -1565919581, 
          lines = 0x28aa7da}
        GDALmetadata = 0x7ffff77aa094
        have_colors = 0
        gdal_rat = 0x7ffff752f25f <find_file1+198>
#5  0x0000000000405231 in main (argc=3, argv=0x7fffffffdad8) at main.c:619
        nBand = 1
        input = 0x627d20 "/mnt/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt"
        output = 0x627dc0 "loss_global_forest_cover_loss_2000_2015"
        title = 0x0
        cellhd = {format = 0, compressed = 0, rows = 560000, rows3 = 560000, cols = 1440000, cols3 = 1440000, depths = 1, proj = 3, zone = 0, 
          ew_res = 0.00025000000000000017, ew_res3 = 0.00025000000000000017, ns_res = 0.00025000000000000017, ns_res3 = 0.00025000000000000017, tb_res = 1, 
          north = 80, south = -60.000000000000085, east = 180.00000000000023, west = -180, top = 1, bottom = 0}
        loc_wind = {format = 0, compressed = -1, rows = 1, rows3 = 1, cols = 1, cols3 = 1, depths = 1, proj = 3, zone = 0, ew_res = 1, ew_res3 = 1, 
          ns_res = 1, ns_res3 = 1, tb_res = 1, north = 1, south = 0, east = 1, west = 0, top = 1, bottom = 0}
        cur_wind = {format = 0, compressed = 0, rows = 0, rows3 = 0, cols = 0, cols3 = 0, depths = 0, proj = 0, zone = 0, ew_res = 0, ew_res3 = 0, 
          ns_res = 0, ns_res3 = 0, tb_res = 0, north = 0, south = 0, east = 0, west = 0, top = 0, bottom = 0}
        proj_info = 0x68ced0
        proj_units = 0x68cbf0
        loc_proj_info = 0x68e5f0
        loc_proj_units = 0x7db4f0
        hDS = 0x7dbb00
        hDriver = 0x62ef90
        hBand = 0x7dbcb0
        adfGeoTransform = {-180, 0.00025000000000000017, 0, 80, 0, -0.00025000000000000017}
        n_bands = 0
        force_imagery = 0
        error_msg = "\253\000\000\000\022", '\000' <repeats 19 times>, "i\000\000\000\022", '\000' <repeats 19 times>, "\222\000\000\000\022", '\000' <repeats 19 times>, "\231\000\000\000\022", '\000' <repeats 19 times>, "\001\000\000\000 \000\000\000@\323\377\377\377\177\000\000\260\327\377\377\377\177\000\000\350
Y\372\367\377\177\000\000\060U\372\367\377\177\000\000\001\000\000\000\000\000\000\000\235\066\313\335\377\177\000\000\260Q\336\367\377\177\000\000\000\000\000\000\000\000\000\000D|\336\367\377\177\000\000`O\372\367\377\177\000\000\210\337\377\367\377\177\000\000\000\000\000\000\000\000\000\000"...
        projcomp_error = 1
        overwrite = 0
        offset = 0
        suffix = 0x629130 ""
        num_digits = 0
        module = 0x7ffff777ea28 <state+40>
        parm = {input = 0x7ffff777ea88 <state+136>, output = 0x626ea0, target = 0x6270e0, title = 0x6271a0, outloc = 0x6274a0, band = 0x626f60, 
          memory = 0x627020, offset = 0x627260, num_digits = 0x627320, map_names_file = 0x6273e0, rat = 0x627560}
        flag_o = 0x7ffff777ea58 <state+88>
        flag_e = 0x626840
        flag_k = 0x627ab0
        flag_f = 0x627a10
        flag_l = 0x627a60
        flag_c = 0x627b00
        flag_p = 0x627b50
        flag_j = 0x626550
(gdb) 

Doing a convert operation with GDAL, no apparent issues (I CTRL-C'd it then):

gdal_translate /mnt/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt /scratch/bla.tif
Input file size is 1440000, 560000
0..

Data source: http://earthenginepartners.appspot.com/science-2013-global-forest/download_v1.3.html

I created a VRT from the "loss" layer tiles which I then tried to import as above:

gdalbuildvrt loss_global_forest_cover_loss_2000_2015.vrt *.tif

gdalinfo loss_global_forest_cover_loss_2000_2015.vrt
Driver: VRT/Virtual Raster
[...]
Size is 1440000, 560000
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433],
    AUTHORITY["EPSG","4326"]]
Origin = (-180.000000000000000,80.000000000000000)
Pixel Size = (0.000250000000000,-0.000250000000000)
Corner Coordinates:
Upper Left  (-180.0000000,  80.0000000) (180d 0' 0.00"W, 80d 0' 0.00"N)
Lower Left  (-180.0000000, -60.0000000) (180d 0' 0.00"W, 60d 0' 0.00"S)
Upper Right ( 180.0000000,  80.0000000) (180d 0' 0.00"E, 80d 0' 0.00"N)
Lower Right ( 180.0000000, -60.0000000) (180d 0' 0.00"E, 60d 0' 0.00"S)
Center      (   0.0000000,  10.0000000) (  0d 0' 0.00"E, 10d 0' 0.00"N)
Band 1 Block=128x128 Type=Byte, ColorInterp=Gray

Test with GRASS GIS 7.3:

GRASS 7.3.svn (latlong):~/software/grass73_svn > r.in.gdal input=/mnt/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt output=loss_global_forest_cover_loss_2000_2015
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
0% Segmentation fault (core dumped)

System:

gcc -v
...
gcc version 6.3.1 20161221 (Red Hat 6.3.1-1) (GCC) 

uname -a 
Linux fedora-calc 4.9.12-200.fc25.x86_64 #1 SMP Thu Feb 23 19:31:49 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Change History (0)

Note: See TracTickets for help on using tickets.