ArcInfo Coverage opening error in OGR module of GDAL

[iprs@…]

When I open any Arc/Info Coverage files
the OGR modules in GDAL returns fatal error.
It is occur when OGR try to open features from PAL (polygon) layer.
In this case (polygon layer opening) the OGR misinterpret PAL
as ARC layer and try to do _AVCBinReadNextArc() in avc_bin.c,
but not _AVCBinReadNextPal() function.
It seams to me, the eTypeFile parameters in _AVCBinReadNextObject()
function take on a AVCFileArc value, but it must be AVCFilePal.

[warmerdam]

Dmitry,

This does not happen with all binary coverages.  Could you attach the
smallest coverage that you have that reproduces this problem?

Thanks

[iprs@…]

ArcInfo Binary Coverage sample with error when PAL opening

[warmerdam]

I have done preliminary testing with the attached layer, and I am not
sure I see Dmitry's problem. 

warmerda@gdal2200[74]% ogrinfo -ro 2
OGR: OGROpen(2) succeeded (0x804caa8).
INFO: Open of `2'
using driver `ESRI Shapefile' successful.
1: AAT (None)
2: BND (None)
3: PAT (None)
4: TIC (None)
warmerda@gdal2200[77]% ogrinfo -ro 2 PAT | more
OGR: OGROpen(2) succeeded (0x804caa8).
INFO: Open of `2'
using driver `ESRI Shapefile' successful.

Layer name: PAT
Geometry: None
Feature Count: 67
Layer SRS WKT:
(unknown)
AREA: Real (13.6)
PERIMETER: Real (13.6)
RIV25M_: Integer (11.0)
RIV25M_ID: Integer (11.0)
TYPE: Integer (1.0)
WATER_TYPE: String (20.0)
COUNTRY: String (2.0)
CNTRY_NAME: String (40.0)
REGION: String (21.0)
CONTINENT: String (13.0)
OGRFeature(PAT):0
  AREA (Real) = -64800.000000
  PERIMETER (Real) =   1080.000000
  RIV25M_ (Integer) = 1
  RIV25M_ID (Integer) = 0
  TYPE (Integer) = 0
  WATER_TYPE (String) = (null)
  COUNTRY (String) = (null)
  CNTRY_NAME (String) = (null)
  REGION (String) = (null)
  CONTINENT (String) = (null)

OGRFeature(PAT):1
  AREA (Real) =  32347.910000
  PERIMETER (Real) =   1019.278000
  RIV25M_ (Integer) = 2
  RIV25M_ID (Integer) = 86
  TYPE (Integer) = 2
  WATER_TYPE (String) = Not inland water
  COUNTRY (String) = (null)
  CNTRY_NAME (String) = (null)
  REGION (String) = (null)
  CONTINENT (String) = (null)

...

However, while PAL access seems to be working there are some issues.
1) The sample dataset includes the data directory but not the info directory.
2) The directory contains a substantial ARC file, but there is no corresponding
ARC layer offered. 

I would note this appears to be an older style of arc/info binary coverage
file, not a Arc/Info 7.x file set.  I believe there is a bug (the lack of
the ARCs if nothing else) but I don't have time to dig into AVC bugs at this
time.  Keeping the bug report on file till someone has time to look at it. 
Adding Daniel as a CC: since AVC is his lib. 

[iprs@…]

Thanks a lot for your help.
Please see, it is not "ESRI Shapefile" as you got from test of folder name.
This is must be ArcInfo Binary Coverage.
I hope, Daniel will fix this problem.

[warmerdam]

Ahh, I see what is going on (to some degree anyways).  In my case the 
directory was being opened as a shapefile dataset because it has some DBF
files in it, instead of going on to the arc/info binary coverage driver.  I 
will diagnose a bit more at my end to see what I can do. 

[iprs@…]

It seems to me the problem in this ideas of OGR
/* -------------------------------------------------------------------- */
/*      If this is a polygon layer, try to assemble the arcs to form    */
/*      the whole polygon geometry.                                     */
/* -------------------------------------------------------------------- */
    if( psSection->eType == AVCFilePAL 
        || psSection->eType == AVCFileRPL )
        FormPolygonGeometry( poFeature, (AVCPal *) pFeature );

[Daniel Morissette]

(For the record, attachment 124 is a zip file)

FYI I tested Dmitry's coverage with the AVCEXPORT utility, and it is able to
read the coverage without errors.  So the problem must be in the way that OGR
uses the AVC lib. Actually, it may simply be that the DBF files confuse OGR to
think that it's a shapefile, as suggested by Frank.

About the missing info directory, this seems to be a PC Arc/Info coverage, and
in that version there was no info directory: the attribute files were in DBFs in
the coverage directory. 
See http://avce00.maptools.org/docs/v7_bin_cover.html#VARIANTS

[iprs@…]

I have tried to execute debuging AVC-PAL import and
it seems to me, the error is in the code of trying reading
of Object by Index from ARX index file.

So... in avc\avc_bin.c module in function
void *AVCBinReadObject(AVCBinFile *psFile, int iObjIndex )
we can see   
     /*-----------------------------------------------------------------
     * Establish the offset to read the object from.
     *----------------------------------------------------------------*/
    if( bIndexed )
    {
        AVCRawBinFSeek( psFile->psIndexFile, 100 + (iObjIndex-1)*8, SEEK_SET );
        if( AVCRawBinEOF( psFile->psIndexFile ) )
            return NULL;

        nObjectOffset = AVCRawBinReadInt32( psFile->psIndexFile );
        ...
The value nObjectOffset is not correct returned.
Could you test this case. Reading ARC by Index.
Not as series of access when nFID == -3, but as
AVCBinReadObject( hFile, nFID );

/* -------------------------------------------------------------------- */
/*      Read the raw feature - the -3 fid is a special flag             */
/*      indicating serial access.                                       */
/* -------------------------------------------------------------------- */
    void *pFeature;

    if( nFID == -3 )
    {
        while( (pFeature = AVCBinReadNextObject( hFile )) != NULL
               && !MatchesSpatialFilter( pFeature ) )
        {
            nNextFID++;
        }
    }
    else
    {
        bNeedReset = TRUE;
        pFeature = AVCBinReadObject( hFile, nFID );
    }


Sorry for my english

[warmerdam]

I would like to correct this before the 1.2.0 release.  Made it release critical.

I would note, it might also be related to #163.

[warmerdam]

Modified ogrshapedatasource.cpp to not treat a directory with an ARC file
as a shapefile coverage if there are no .shp files in it. 

[warmerdam]

OK, as Dmitry identified, the problem was with reading features using
the index.  The index code I implemented in AVCBinReadObject() did not
actually work properly for the PC Arc/Info format with the extra 256 byte
headers.  I have corrected that now in the upstream avc_bin.c code, and updated
GDAL with the latest. 

The coverage now seems to work fine!

Thanks for finding, and helping diagnose this bug Dmitry.  Of course, 
PC Arc/Info format is pretty ancient.  Hopefully we won't encounter too
many of the datasets in the future, but at least now it works properly in
OGR.