Opened 11 years ago

Closed 11 years ago

Last modified 11 years ago

#1534 closed defect (fixed)

OGR GML reader fails if file has UTF-8 BOM prefix

Reported by: rogerjames99 Owned by: Mateusz Łoskot
Priority: normal Milestone: 1.4.1
Component: OGR_SF Version: unspecified
Severity: normal Keywords: UTF BOM GML
Cc: warmerdam

Description (last modified by warmerdam)

The function OGRGMLDataSource::Open in ogrgmldatasource.cpp fails is the GML file has a UTF-8 encoded UNICODE BOM (Byte order mark) at the start of the file. This is valid UTF-8 encoding (see RFC 3629 section 6) and should be allowed. Xerces properly handles this sequence. The code below is a modification to this function to allow for this.

It may be better to remove the "Test Open" functionality altogether and just let Xerces worry about correctly formed xml.

int OGRGMLDataSource::Open( const char * pszNewName, int bTestOpen )

    FILE        *fp;
    char        szHeader[1000];

/* -------------------------------------------------------------------- */
/*      Open the source file.                                           */
/* -------------------------------------------------------------------- */
    fp = VSIFOpen( pszNewName, "r" );
    if( fp == NULL )
        if( !bTestOpen )
            CPLError( CE_Failure, CPLE_OpenFailed, 
                      "Failed to open GML file `%s'.", 
                      pszNewName );

        return FALSE;

/* -------------------------------------------------------------------- */
/*      If we aren't sure it is GML, load a header chunk and check      */
/*      for signs it is GML                                             */
/* -------------------------------------------------------------------- */
    if( bTestOpen )
		char *szPtr = szHeader;
        VSIFRead( szHeader, 1, sizeof(szHeader), fp );
        szHeader[sizeof(szHeader)-1] = '\0';
/* -------------------------------------------------------------------- */
/*      Check for a UTF-8 BOM and skip if found                         */
/* -------------------------------------------------------------------- */
		if (((unsigned char)szPtr[0] == 0xEF) && ((unsigned char)szPtr[1] == 0xBB) && ((unsigned char)szPtr[2] == 0xBF))
			szPtr += 3;

        if( szPtr[0] != '<' 
            || strstr(szPtr,"") == NULL )
            VSIFClose( fp );
            return FALSE;

Attachments (2)

ogrgmldatasource.cpp (32.5 KB) - added by rogerjames99 11 years ago.
smallsample.gml (3.7 KB) - added by rogerjames99 11 years ago.

Download all attachments as: .zip

Change History (8)

Changed 11 years ago by rogerjames99

Attachment: ogrgmldatasource.cpp added

comment:2 Changed 11 years ago by rogerjames99

Component: defaultOGR_SF

Sorry about the typos and munged source. I have attached the modded file from the 1.4.0 source.

comment:3 Changed 11 years ago by warmerdam

Cc: warmerdam added
Description: modified (diff)
Milestone: 1.4.1
Owner: changed from warmerdam to Mateusz Łoskot
Priority: lownormal
Severity: minornormal


Could you attach a smallish file with this marker in it?


Could you fix this trunk and 1.4 branch? We *do* want to keep the pre-checks but they need to be safer. We don't want to start up xerces for every file passed to OGROpen().

Please add a test for this in the test suite.

Changed 11 years ago by rogerjames99

Attachment: smallsample.gml added

comment:4 Changed 11 years ago by rogerjames99


smallsample.gml attached.


comment:5 Changed 11 years ago by Mateusz Łoskot

Status: newassigned

The fix proposed in attached patch provides support of BOM indicator only for UTF-8 encoding. BOM is a variable-length mark and depends on actual encoding of XML file. In future, it's reasonable to add support for other XML encodings.

comment:6 Changed 11 years ago by Mateusz Łoskot

Resolution: fixed
Status: assignedclosed

The fix has been applied (r11191).

Also, test suite has been updated with new case reading GML with BOM indicator (bom.gml).

comment:7 Changed 11 years ago by Mateusz Łoskot

The fix has been also submitted to stable branch 1.4 (r11193)

Note: See TracTickets for help on using tickets.