OGR .IND files not used if read-only

[Daniel Morissette]

This is indirectly related to #839. 

I found that if the .IND file or the whole set of shapefiles for a given dataset
are read-only, then the .IND file is not used. Just making the .ind file
writeable is enough to have it used.

$ chmod -w shptest2.*
$ ogrinfo shptest2.shp -sql "SELECT * FROM shptest2 WHERE parcelid = 'D0134 
M00015'"
ERROR 4: Unable to open shptest2.shp or shptest2.SHP.
ERROR 3: Open() failed for shptest2.ind
ERROR 4: Failed to open index file shptest2.ind.
Had to open data source read-only.
...
... the query works but doesn't use the index (slow)
...

$ chmod +w shptest2.ind
$ ls -l shptest2.*
-r--r--r--    1 daniel   users    276714879 Apr 27 11:07 shptest2.dbf
-r--r--r--    1 daniel   users         227 Apr 27 11:09 shptest2.idm
-rw-rw-r--    1 daniel   users    17092608 Apr 29 11:06 shptest2.ind
-r--r--r--    1 daniel   users          38 Apr 27 11:05 shptest2.prj
-r--r--r--    1 daniel   users    307403144 Apr 27 11:07 shptest2.shp
-r--r--r--    1 daniel   users     2657620 Apr 27 11:07 shptest2.shx
$ ogrinfo shptest2.shp -sql "SELECT * FROM shptest2 WHERE parcelid = 'D0134 
M00015'"
ERROR 4: Unable to open shptest2.shp or shptest2.SHP.
Had to open data source read-only.
...
... this time the index is being used and returns fast
...

[Daniel Morissette]

Looking at this now.

[warmerdam]

Daniel, 

I imagine the problem is this code in
gdal/ogr/ogrsf_frmts/generic/ogr_miattrind.cpp.
I suspect it ought to try and open the .ind file read-only 
if r+ fails (and be silent about the first error).  It might
be necessary to keep track that the index is readonly so we 
know we can't update it, though I am pretty sure we don't keep
the index up to date in the face of updates anyways.


/* -------------------------------------------------------------------- */
/*      Open the index file.                                            */
/* -------------------------------------------------------------------- */
    poINDFile = new TABINDFile();
    if( poINDFile->Open( pszMetadataFilename, "r+" ) != 0 )
    {
        CPLDestroyXMLNode( psRoot );
        CPLError( CE_Failure, CPLE_OpenFailed,
                  "Failed to open index file %s.", 
                  pszMIINDFilename );
        return OGRERR_FAILURE;
    }

[Daniel Morissette]

Thanks for the hint Frank. I suspected it was something like that. I got pulled
away to other stuff after creating the bug this morning and had not really
looked yet. I'll try this now.

[Daniel Morissette]

Falling back on "r" access if "r+" failed solved my problem. The
TABINDFile::Open() method already had a 3rd optional argument to be silent if
the open failed. 

diff -u -r1.4 ogr_miattrind.cpp
--- ogr_miattrind.cpp   2 Dec 2003 19:59:39 -0000       1.4
+++ ogr_miattrind.cpp   29 Apr 2005 17:30:54 -0000
@@ -240,7 +240,8 @@
 /*      Open the index file.                                            */
 /* -------------------------------------------------------------------- */
     poINDFile = new TABINDFile();
-    if( poINDFile->Open( pszMetadataFilename, "r+" ) != 0 )
+    if(  poINDFile->Open( pszMetadataFilename, "r+", TRUE ) != 0 &&
+         poINDFile->Open( pszMetadataFilename, "r" ) != 0 )
     {
         CPLDestroyXMLNode( psRoot );
         CPLError( CE_Failure, CPLE_OpenFailed,


I didn't commit anything to CVS yet because I think it would be best to
propagate and use the same access mode as was used to open the main file, but
I'm a bit lost in the stack of calls in OGR that get us to this location in the
code, and I'm unable to build GDAL any more on my local machine at the moment.
I'm also not sure if you generally track that kind of stuff in other drivers and
if it's worth worrying about this here.

About tracking whether the index is read-only, should we do that as a boolean in
the OGRMIAttrIndex class or do you have another preference?

[Jukka Rahkonen]

Can this still be a valid ticket after 10 years with no activity?

[Daniel Morissette]

We can probably close as wontfix since this is a very specific corner case, and the fact that nobody complained in 10 years shows how (not) important it is for general use.