ogr2ogr, shp > gml: incomplete schema

[peifer]

2 attributes (UPDATE_WHE and INS_WHEN) seem to be missing in the schema file, which is attached.

Here some info from the shapefile:

$ ogrinfo -al -so RBD_F1v3.shp
INFO: Open of `RBD_F1v3.shp'
      using driver `ESRI Shapefile' successful.

Layer name: RBD_F1v3
Geometry: Polygon
Feature Count: 215
Extent: (-2823913.983300, -3076144.333200) - (10026124.658800, 5418276.299000)
Layer SRS WKT:
PROJCS["ETRS_1989_LAEA_52N_10E",
    GEOGCS["GCS_ETRS_1989",
        DATUM["European_Terrestrial_Reference_System_1989",
            SPHEROID["GRS_1980",6378137.0,298.257222101]],
        PRIMEM["Greenwich",0.0],
        UNIT["Degree",0.0174532925199433]],
    PROJECTION["Lambert_Azimuthal_Equal_Area"],
    PARAMETER["False_Easting",4321000.0],
    PARAMETER["False_Northing",3210000.0],
    PARAMETER["Central_Meridian",10.0],
    PARAMETER["Latitude_Of_Origin",52.0],
    UNIT["Meter",1.0]]
NAME: String (100.0)
LGE_ID: String (2.0)
CTY_ID: String (2.0)
NAME_ENGL: String (100.0)
AREAKM2: Real (19.11)
EUCD_RBD: String (50.0)
EUCD_NATRB: String (20.0)
MSCD_RBD: String (25.0)
intCode: String (25.0)
HYD_SYST_O: String (50.0)
HYD_SYST_S: String (50.0)
ISLANDS: String (50.0)
RBD_HYCODE: String (25.0)
natRBD_HYC: String (50.0)
SEA: String (50.0)
INTERNATIO: String (5.0)
EU_MS: String (5.0)
UPDATE_WHE: Date (10.0)
UPDATE_BY: String (50.0)
META_ID: String (50.0)
INS_WHEN: Date (10.0)
INS_BY: String (50.0)
ID: Real (19.11)
Viewer_CD: String (25.0)
Shape_Leng: Real (19.11)
Shape_Area: Real (19.11)

[peifer]

Sorry, I forgot to mention that I was actually running this command:

$ ogr2ogr -f GML -t_srs EPSG:4326 RBD_F1v3_ng.gml RBD_F1v3.shp

... and my version is:

$ ogr2ogr --version
GDAL 1.6.0, released 2008/12/04

[warmerdam]

I will need the input shapefile in order to pursue this problem.

[peifer]

Replying to warmerdam:

I will need the input shapefile in order to pursue this problem.

It is the last entry under GIS data:
Zipped shapefile with RBDs, version 1.3, polygon, vector

​http://dataservice.eea.europa.eu/dataservice/metadetails.asp?id=1041

[Even Rouault]

The 2 missing fields are of type OFT_Date and by reading OGRGMLDataSource::InsertHeader(), it appears that only OFT_Integer, OFT_Real and OFT_String are handled currently.

[warmerdam]

I have confirmed that the fields named intCode and META_CODE are not reported by the OGR driver opening the GML file. The actual values do appear to be in the GML file (and in the .xsd file). The issue seems to be that both these fields are null on the first feature encountered so the scanner does not add them to the schema.

It does not *appear* that this problem relates to date fields (which are currently translated adequately as strings) or to the .xsd GML writer. Just to the GML reader/scanner.

Turning over to Chaitanya to dig more deeply into and to recommend a fix.

[peifer]

Replying to warmerdam:

I have confirmed that the fields named intCode and META_CODE are not reported by the OGR driver opening the GML file. The actual values do appear to be in the GML file (and in the .xsd file). The issue seems to be that both these fields are null on the first feature encountered so the scanner does not add them to the schema.

Thanks for the detailed follow-up. By the way: it should read META_ID, rather than META_CODE.

Just a few, potentially unhelpful thoughts from a non-programmer perspective:

ogrinfo seems to have a clear idea that there are 26 fields in the dbf file. This is also what other command line tools (dbfinfo, dbfdump) are telling me. Here the output from orginfo, which is also at the start of the bug report.

     1  NAME: String (100.0)
     2  LGE_ID: String (2.0)
     3  CTY_ID: String (2.0)
     4  NAME_ENGL: String (100.0)
     5  AREAKM2: Real (19.11)
     6  EUCD_RBD: String (50.0)
     7  EUCD_NATRB: String (20.0)
     8  MSCD_RBD: String (25.0)
     9  intCode: String (25.0)
    10  HYD_SYST_O: String (50.0)
    11  HYD_SYST_S: String (50.0)
    12  ISLANDS: String (50.0)
    13  RBD_HYCODE: String (25.0)
    14  natRBD_HYC: String (50.0)
    15  SEA: String (50.0)
    16  INTERNATIO: String (5.0)
    17  EU_MS: String (5.0)
    18  UPDATE_WHE: Date (10.0)
    19  UPDATE_BY: String (50.0)
    20  META_ID: String (50.0)
    21  INS_WHEN: Date (10.0)
    22  INS_BY: String (50.0)
    23  ID: Real (19.11)
    24  Viewer_CD: String (25.0)
    25  Shape_Leng: Real (19.11)
    26  Shape_Area: Real (19.11)

It looks to me that once this is clear, the subsequent creation of xsd, gml and kml file should just blindly follow the field structure. I wouldn't know why you have to test for NULL values each time.

<ogr:META_ID></ogr:META_ID>              # gml export, or 
<SimpleData name="META_ID"></SimpleData> # kml export

Empty elements in the gml and kml exports like in the example above would be much more informative than missing elements (which can mean anything, including a scanner bug, or whatever).

[chaitanya]

Replying to warmerdam:

It does not *appear* that this problem relates to date fields (which are currently translated adequately as strings) or to the .xsd GML writer. Just to the GML reader/scanner.

Actually, the problem is with the .xsd writer in ogrgmldatasource.cpp . I am attaching a patch which has a fix for this.

I have also noted that the precision of the decimal data types is not considered by the GML driver. This patch also fixes some issues with the handling of the precision of OFT_Real datatype.

I am going to apply this patch in the trunk after creating some tests.

[chaitanya]

Patch for fixing the handling of date types and precision data for decimal types

[chaitanya]

Applied the patch and a test to the trunk in r17629 and to the 1.6 branch in r17630.

Future work might include support for some other field types like OFTDate and OFTStringList.