Shapefile reader losing last record with some some VSI implementations

[warmerdam]

When a vendor custom VSI driver is used with the OGR shapefile driver, the last feature is often discarded due to this logic in ogrshapelayer.cpp:

                else if( VSIFEofL((VSILFILE*)hDBF->fp) )
                    return NULL; /* There's an I/O error */

This code seems to depend on VSIFEofL() only returning TRUE if reading has attempted to go beyond the end of the file, while the custom VSI implementation is unable to distinguish this case from the case of the file pointer being at the end of the file (but not beyond).

By POSIX logic it may be that the custom driver is techically wrong, but this dependence on a subtle interpretation of file io semantics seems unnecessarily fragile, so I'd like to consider how the goal can be accomplished in a less fragile way.

[Even Rouault]

Without seeing the code, it is a bit difficult for me to imagine why the custom VSI implementation cannot be simply fixed to set the bEOF flag in the Read() method when it is called with nOffset >= nFileSize, and the Eof() method would just return the bEOF flag.

The alternative, a local fix in the shapefile driver, would be to be able to detect that DBFLoadRecord() has failed in a FSeek() or FRead() call. A call to CPLGetLastErrorType() could perhaps do that, with a CPLErrorReset() just before calling DBFIsRecordDeleted() at the both places where it is called in ogrshapelayer.cpp. Not very pretty however.

Or more simple instead of calling directly DBFIsRecordDeleted(), we could call DBFLoadRecord(), and just test its return value. But DBFLoadRecord() is currently static in dbfopen.c

[Jukka Rahkonen]

Is this related with http://trac.osgeo.org/gdal/ticket/5093 and fixed at the same? Both handle the last record of shapefile, vsi and POSIX.

[Jukka Rahkonen]

I repeat my question, is this a duplicate of #5093?

[Even Rouault]

There's connection. #5093 was actually fixing a similar defect in in-tree VSI implementation. This ticket is about out-of-tree non-POSIX compliant VSI implementation.

[Even Rouault]

Closing with the assumption there's no longer interest in that