Opened 8 years ago

Closed 3 years ago

Last modified 3 years ago

#3031 closed defect (fixed)

Attempt to read past EOF on AIG Windows VAT table

Reported by: grovduck Owned by: dmorissette
Priority: normal Milestone: 1.11.0
Component: GDAL_Raster Version: svn-trunk
Severity: minor Keywords: ReadRAT, EOF, AVC
Cc: matt.gregory@…, jason.roberts@…, neteler, maphew@…, eadam@…, ilucena, warmerdam, chaitanya

Description

Using svn-trunk on Windows XP with ArcInfo? 9.3 grids. When I try to open grids, I receive this message:

ERROR 3: Attempt to read past EOF in D:/matt/vat_test/grid1/../info/../grid1/vat.adf

when it calls ReadRAT() in aigdataset.cpp. I've tested using two different grids, one with 1280 unique values with VALUE and COUNT fields and then a clip of the first with 398 unique values with VALUE and COUNT. Test 1 gives the error message, test 2 is fine.

As in ticket #2447, the attribute data actually reads in correctly, so this is more of an annoyance than a major issue.

Test grids are at: ftp://ftp.fsl.orst.edu/pub/gregorym/vat_test.zip (53.4 MB)

Attachments (1)

clip.zip (512.3 KB) - added by EliL 3 years ago.
small attachment that produces error for #3031

Download all attachments as: .zip

Change History (21)

comment:1 Changed 8 years ago by Even Rouault

Keywords: AVC added
Owner: changed from warmerdam to dmorissette

Assigning to Daniel. The error is issued by the AVC driver

comment:2 Changed 8 years ago by neteler

See also trac #2447

comment:3 Changed 8 years ago by Even Rouault

Cc: matt.gregory@… jason.roberts@… neteler@… added

Daniel,

I've pushed in trunk (r17430) and branches/1.6 (r17431) a workaround to enable users to open those datasets that trigger the error with binding languages. Open() in swig/include/gdal.i checks that there is no failure returned by CPLGetLastErrorType() even if the dataset is not NULL. I've tried to look a bit at why the error was issued in the AVC driver but it didn't seem obvious how to fix that without knowledge of the format.

comment:4 Changed 8 years ago by Even Rouault

r17432 (trunk only) : differ opening of the RAT only when GetDefaultRat?() is called. Will improve performances and make less likely the error message of #3031

comment:5 Changed 8 years ago by grovduck

comment:6 Changed 7 years ago by maphew

Cc: maphew@… added

I get this same error on 2 AIG datasets from a set of 60. They were generated by Arcinfo Workstation (GRID) but at different times (2006 and 2007) and perhaps different versions, in the 9.x range though.

Dataset at ftp://ftp.geomaticsyukon.ca/users/matt/AIG_read_past_EOF.7z 25mb.

>gdalinfo --version
GDAL 1.7.0dev, released 2008/11/26

>for /d %a in (dem_105e dem_105d dem_105o) do gdal_translate %a gdalout\%a.tif

>gdal_translate dem_105e gdalout\dem_105e.tif
Input file size is 3701, 3857
0...10...20...30...40...50...60...70...80...90...100 - done.

>gdal_translate dem_105d gdalout\dem_105d.tif
Input file size is 3817, 3858
0ERROR 3: Attempt to read past EOF in dem_105d/../info/arc.dir.
ERROR 4: Failed to open table dem_105d.VAT
...10...20...30...40...50...60...70...80...90...100 - done.

>gdal_translate dem_105o gdalout\dem_105o.tif
Input file size is 3410, 3804
0ERROR 3: Attempt to read past EOF in dem_105o/../info/../dem_105o/vat.adf.
...10...20...30...40...50...60...70...80...90...100 - done.

comment:7 Changed 7 years ago by EliL

Cc: eadam@… added

comment:8 Changed 7 years ago by neteler

Cc: neteler added; neteler@… removed
Milestone: 1.7.1

The problem is still there in GDAL 1.7.svn (using a 64bit Linux box):

gdalinfo $INPUT/dsm000038/                                                                   
Driver: AIG/Arc/Info Binary Grid
Files: /geodata/public/pat_DTM_DSM_LIDAR_2009_UTM_WGS84_dati_grezzi/DSM_BUFF_GRD/dsm_orto_grd1/dsm000038/
       /geodata/public/pat_DTM_DSM_LIDAR_2009_UTM_WGS84_dati_grezzi/DSM_BUFF_GRD/dsm_orto_grd1/dsm000038/w001001x.adf
       /geodata/public/pat_DTM_DSM_LIDAR_2009_UTM_WGS84_dati_grezzi/DSM_BUFF_GRD/dsm_orto_grd1/dsm000038/sta.adf
       /geodata/public/pat_DTM_DSM_LIDAR_2009_UTM_WGS84_dati_grezzi/DSM_BUFF_GRD/dsm_orto_grd1/dsm000038/w001001.adf
       /geodata/public/pat_DTM_DSM_LIDAR_2009_UTM_WGS84_dati_grezzi/DSM_BUFF_GRD/dsm_orto_grd1/dsm000038/log
       /geodata/public/pat_DTM_DSM_LIDAR_2009_UTM_WGS84_dati_grezzi/DSM_BUFF_GRD/dsm_orto_grd1/dsm000038/dblbnd.adf
       /geodata/public/pat_DTM_DSM_LIDAR_2009_UTM_WGS84_dati_grezzi/DSM_BUFF_GRD/dsm_orto_grd1/dsm000038/hdr.adf
Size is 2020, 2020
Coordinate System is `'
Origin = (713990.000000000000000,5152010.000000000000000)
Pixel Size = (1.000000000000000,-1.000000000000000)
Corner Coordinates:
Upper Left  (  713990.000, 5152010.000)
Lower Left  (  713990.000, 5149990.000)
Upper Right (  716010.000, 5152010.000)
Lower Right (  716010.000, 5149990.000)
Center      (  715000.000, 5151000.000)
Band 1 Block=505x4 Type=Float32, ColorInterp=Undefined
  Min=1730.546 Max=2535.939
  NoData Value=-3.4028234663852886e+38
ERROR 3: Attempt to read past EOF in /geodata/public/pat_DTM_DSM_LIDAR_2009_UTM_WGS84_dati_grezzi/DSM_BUFF_GRD/dsm_orto_grd1/dsm000038//../info/arc.dir.
ERROR 4: Failed to open table .VAT

I wonder if it is harmless or harmful.

comment:9 Changed 5 years ago by ilucena

Cc: ilucena added

comment:10 Changed 5 years ago by chaitanya

Owner: changed from dmorissette to chaitanya
Status: newassigned

comment:11 Changed 5 years ago by chaitanya

Resolution: fixed
Status: assignedclosed

This message is shown only when the file length of vat.adf is a multiple of 1024. The error message is displayed because the the end-of-file test failed at the end of the buffer.

I also noticed that the driver does not report/recover if the file ends abruptly in the middle of a record.

Fixed in trunk (r24578) and 1.9 branch (r24579).

comment:12 Changed 5 years ago by warmerdam

Resolution: fixed
Status: closedreopened

comment:13 Changed 5 years ago by warmerdam

Cc: warmerdam chaitanya added
Owner: changed from chaitanya to dmorissette
Status: reopenednew

Good work on the fix! This code comes from an upstream source, so I'm reopening the ticket and assigning it to Daniel to upstream the changes.

comment:14 in reply to:  8 ; Changed 3 years ago by peifer

Replying to neteler:

The problem is still there in GDAL 1.7.svn (using a 64bit Linux box):

(...)

> ERROR 3: Attempt to read past EOF in /geodata/public/pat_DTM_DSM_LIDAR_2009_UTM_WGS84_dati_grezzi/DSM_BUFF_GRD/dsm_orto_grd1/dsm000038//../info/arc.dir.
> ERROR 4: Failed to open table .VAT

I am getting this error (now with a somewhat different wording) when using GDAL from trunk.

ERROR 3: EOF encountered in path/to/soil01/../info/arc.dir after reading 0 bytes while trying to read 32 bytes. File may be corrupt.
ERROR 4: Failed to open table soil01.VAT

As far as I can see: the error only occurs for Float32 rasters. None of them has a vat.adf file. The error does not occur when using gdalinfo on my Int16 rasters (which all have a vat.adf file). Using the -norat switch avoids the error.

comment:15 in reply to:  14 ; Changed 3 years ago by Even Rouault

Replying to peifer:

I am getting this error (now with a somewhat different wording) when using GDAL from trunk.

ERROR 3: EOF encountered in path/to/soil01/../info/arc.dir after reading 0 bytes while trying to read 32 bytes. File may be corrupt.
ERROR 4: Failed to open table soil01.VAT

For the sake of completness, it would be good if you could attach a dataset that causes issues to the ticket. (concerning the different wording of the error message, it is in fact the same error. The wording was changed in r24578)

comment:16 in reply to:  15 Changed 3 years ago by peifer

Replying to rouault:

For the sake of completness, it would be good if you could attach a dataset that causes issues to the ticket. (concerning the different wording of the error message, it is in fact the same error. The wording was changed in r24578)

An attachment would be too big for trac, so I uploaded some test data here: http://tinyurl.com/qyvnbvt

soil00 is an Int16 raster where gdalinfo works fine. soil01 is a Float32 raster where the mentioned ERROR 3 and ERROR 4 messages are thrown. I am using GDAL from svn on a 64-bit Linux box.

Changed 3 years ago by EliL

Attachment: clip.zip added

small attachment that produces error for #3031

comment:17 Changed 3 years ago by Even Rouault

Milestone: 1.8.12.0
Resolution: fixed
Status: newclosed

r26515 "AIGRID: Turn off errors that can be triggered if the info has no VAT table related with this coverage (#3031)"

comment:18 Changed 3 years ago by neteler

Does it also fix trac #2447 ?

comment:19 in reply to:  18 Changed 3 years ago by Even Rouault

Replying to neteler:

Does it also fix trac #2447 ?

Perhaps... The best is that you try with your dataset. It just silents an error when there's no .vat to read.

comment:20 Changed 3 years ago by Even Rouault

Milestone: 2.01.11.0
Note: See TracTickets for help on using tickets.