Opened 13 years ago

Closed 12 years ago

#2857 closed defect (fixed)

ogr2ogr, shp > gml: incomplete schema

Reported by: peifer Owned by: chaitanya
Priority: normal Milestone:
Component: OGR_SF Version: 1.6.0
Severity: normal Keywords: GML
Cc: warmerdam

Description

2 attributes (UPDATE_WHE and INS_WHEN) seem to be missing in the schema file, which is attached.

Here some info from the shapefile:

$ ogrinfo -al -so RBD_F1v3.shp
INFO: Open of `RBD_F1v3.shp'
      using driver `ESRI Shapefile' successful.

Layer name: RBD_F1v3
Geometry: Polygon
Feature Count: 215
Extent: (-2823913.983300, -3076144.333200) - (10026124.658800, 5418276.299000)
Layer SRS WKT:
PROJCS["ETRS_1989_LAEA_52N_10E",
    GEOGCS["GCS_ETRS_1989",
        DATUM["European_Terrestrial_Reference_System_1989",
            SPHEROID["GRS_1980",6378137.0,298.257222101]],
        PRIMEM["Greenwich",0.0],
        UNIT["Degree",0.0174532925199433]],
    PROJECTION["Lambert_Azimuthal_Equal_Area"],
    PARAMETER["False_Easting",4321000.0],
    PARAMETER["False_Northing",3210000.0],
    PARAMETER["Central_Meridian",10.0],
    PARAMETER["Latitude_Of_Origin",52.0],
    UNIT["Meter",1.0]]
NAME: String (100.0)
LGE_ID: String (2.0)
CTY_ID: String (2.0)
NAME_ENGL: String (100.0)
AREAKM2: Real (19.11)
EUCD_RBD: String (50.0)
EUCD_NATRB: String (20.0)
MSCD_RBD: String (25.0)
intCode: String (25.0)
HYD_SYST_O: String (50.0)
HYD_SYST_S: String (50.0)
ISLANDS: String (50.0)
RBD_HYCODE: String (25.0)
natRBD_HYC: String (50.0)
SEA: String (50.0)
INTERNATIO: String (5.0)
EU_MS: String (5.0)
UPDATE_WHE: Date (10.0)
UPDATE_BY: String (50.0)
META_ID: String (50.0)
INS_WHEN: Date (10.0)
INS_BY: String (50.0)
ID: Real (19.11)
Viewer_CD: String (25.0)
Shape_Leng: Real (19.11)
Shape_Area: Real (19.11)

Attachments (2)

RBD_F1v3_ng.xsd (7.1 KB) - added by peifer 13 years ago.
2857patch.diff (7.0 KB) - added by chaitanya 12 years ago.
Patch for fixing the handling of date types and precision data for decimal types

Download all attachments as: .zip

Change History (10)

Changed 13 years ago by peifer

Attachment: RBD_F1v3_ng.xsd added

comment:1 Changed 13 years ago by peifer

Sorry, I forgot to mention that I was actually running this command:

$ ogr2ogr -f GML -t_srs EPSG:4326 RBD_F1v3_ng.gml RBD_F1v3.shp

... and my version is:

$ ogr2ogr --version
GDAL 1.6.0, released 2008/12/04

comment:2 Changed 13 years ago by warmerdam

Component: defaultOGR_SF
Keywords: GML added

I will need the input shapefile in order to pursue this problem.

comment:3 in reply to:  2 Changed 13 years ago by peifer

Replying to warmerdam:

I will need the input shapefile in order to pursue this problem.


It is the last entry under GIS data:
Zipped shapefile with RBDs, version 1.3, polygon, vector

http://dataservice.eea.europa.eu/dataservice/metadetails.asp?id=1041

comment:4 Changed 13 years ago by Even Rouault

The 2 missing fields are of type OFT_Date and by reading OGRGMLDataSource::InsertHeader?(), it appears that only OFT_Integer, OFT_Real and OFT_String are handled currently.

comment:5 Changed 13 years ago by warmerdam

Cc: warmerdam added
Owner: changed from warmerdam to chaitanya

I have confirmed that the fields named intCode and META_CODE are not reported by the OGR driver opening the GML file. The actual values do appear to be in the GML file (and in the .xsd file). The issue seems to be that both these fields are null on the first feature encountered so the scanner does not add them to the schema.

It does not *appear* that this problem relates to date fields (which are currently translated adequately as strings) or to the .xsd GML writer. Just to the GML reader/scanner.

Turning over to Chaitanya to dig more deeply into and to recommend a fix.

comment:6 in reply to:  5 Changed 13 years ago by peifer

Replying to warmerdam:

I have confirmed that the fields named intCode and META_CODE are not reported by the OGR driver opening the GML file. The actual values do appear to be in the GML file (and in the .xsd file). The issue seems to be that both these fields are null on the first feature encountered so the scanner does not add them to the schema.

Thanks for the detailed follow-up. By the way: it should read META_ID, rather than META_CODE.

Just a few, potentially unhelpful thoughts from a non-programmer perspective:

ogrinfo seems to have a clear idea that there are 26 fields in the dbf file. This is also what other command line tools (dbfinfo, dbfdump) are telling me. Here the output from orginfo, which is also at the start of the bug report.

     1  NAME: String (100.0)
     2  LGE_ID: String (2.0)
     3  CTY_ID: String (2.0)
     4  NAME_ENGL: String (100.0)
     5  AREAKM2: Real (19.11)
     6  EUCD_RBD: String (50.0)
     7  EUCD_NATRB: String (20.0)
     8  MSCD_RBD: String (25.0)
     9  intCode: String (25.0)
    10  HYD_SYST_O: String (50.0)
    11  HYD_SYST_S: String (50.0)
    12  ISLANDS: String (50.0)
    13  RBD_HYCODE: String (25.0)
    14  natRBD_HYC: String (50.0)
    15  SEA: String (50.0)
    16  INTERNATIO: String (5.0)
    17  EU_MS: String (5.0)
    18  UPDATE_WHE: Date (10.0)
    19  UPDATE_BY: String (50.0)
    20  META_ID: String (50.0)
    21  INS_WHEN: Date (10.0)
    22  INS_BY: String (50.0)
    23  ID: Real (19.11)
    24  Viewer_CD: String (25.0)
    25  Shape_Leng: Real (19.11)
    26  Shape_Area: Real (19.11)

It looks to me that once this is clear, the subsequent creation of xsd, gml and kml file should just blindly follow the field structure. I wouldn't know why you have to test for NULL values each time.

<ogr:META_ID></ogr:META_ID>              # gml export, or 
<SimpleData name="META_ID"></SimpleData> # kml export

Empty elements in the gml and kml exports like in the example above would be much more informative than missing elements (which can mean anything, including a scanner bug, or whatever).

comment:7 in reply to:  5 Changed 12 years ago by chaitanya

Status: newassigned

Replying to warmerdam:

It does not *appear* that this problem relates to date fields (which are currently translated adequately as strings) or to the .xsd GML writer. Just to the GML reader/scanner.

Actually, the problem is with the .xsd writer in ogrgmldatasource.cpp . I am attaching a patch which has a fix for this.

I have also noted that the precision of the decimal data types is not considered by the GML driver. This patch also fixes some issues with the handling of the precision of OFT_Real datatype.

I am going to apply this patch in the trunk after creating some tests.

Changed 12 years ago by chaitanya

Attachment: 2857patch.diff added

Patch for fixing the handling of date types and precision data for decimal types

comment:8 Changed 12 years ago by chaitanya

Resolution: fixed
Status: assignedclosed

Applied the patch and a test to the trunk in r17629 and to the 1.6 branch in r17630.

Future work might include support for some other field types like OFTDate and OFTStringList.

Note: See TracTickets for help on using tickets.