Opened 16 years ago

Closed 16 years ago

#2141 closed defect (fixed)

[PATCH] ogr2ogr truncating data in date element from GML source

Reported by: c911469 Owned by: Even Rouault
Priority: normal Milestone: 1.6.0
Component: OGR_SF Version: unspecified
Severity: normal Keywords: gml
Cc: mlostkot, Even Rouault, warmerdam

Description

I am converting a GML source data file to an ESRI Shapefile and in the process, one of the attribute/elements ("Date" field) is being truncated. It appears also that the data type of the element is being converted from "xs:string" to "integer". In the original GML file, the date might be specified as "2008-01-07" (year-month-day), but in the output SHP file, the result is "2008" (as an integer.) Note also that the "Time" element is being similarly treated is being converted to an integer although no effective information is being lost as "0300" is converted to "300". The ogr2ogr command being used is as follows: ogr2ogr -f "ESRI Shapefile" Tropical_20080106_1500.shp Tropical_20080106_1500.gml

Note also, that ogrinfo has the same affect and incorrectly displays the date fields as described. Command line being used is: ogrinfo -al -ro Tropical_20080106_1500.gml

The FWTools version is 2.0.3 and is the Win32 version. The datafiles utilised are attached, including the ".gfs" file that is created during the running of ogr2ogr.

Attachments (5)

Tropical_20080106_1500.gml (2.1 KB ) - added by c911469 16 years ago.
gml data file
Tropical_20080106_1500.xsd (5.0 KB ) - added by c911469 16 years ago.
xsd file (part of original gml data set)
Tropical_20080106_1500.gfs (2.2 KB ) - added by c911469 16 years ago.
gfs file, created after running ogr2ogr
gdal_svn_trunk_bug2141.patch (5.1 KB ) - added by Even Rouault 16 years ago.
cpl_test_get_value.c (1.2 KB ) - added by Even Rouault 16 years ago.
Small test program for CPLGetValueType

Download all attachments as: .zip

Change History (12)

by c911469, 16 years ago

Attachment: Tropical_20080106_1500.gml added

gml data file

by c911469, 16 years ago

Attachment: Tropical_20080106_1500.xsd added

xsd file (part of original gml data set)

by c911469, 16 years ago

Attachment: Tropical_20080106_1500.gfs added

gfs file, created after running ogr2ogr

comment:1 by Mateusz Łoskot, 16 years ago

I can confirm that date/time values are incorrectly recognized by GML driver. The problem is in GMLPropertyDefn::AnalyzePropertyValue function where date/time value is recognized this way:

  1. digits or hyphen (-) found (line 102) and leave type of a value determined as as untyped

  1. move to condition line 125 which results in true
  1. bIsReal is false, so final result type determined is integer (line 130)

comment:2 by Even Rouault, 16 years ago

Summary: ogr2ogr truncating data in date element from GML source[PATCH] ogr2ogr truncating data in date element from GML source

I'm attaching a patch that addresses the problem in the analysis of strings. I introduce a new CPL function, CPLGetValueType, which returns if a given string is a real, an integer or a string. I think this may be a useful CPL addition as I remember having written similar function in the GPX driver, and I think I took the inspiration from the KML driver... In fact, in the GPX driver, I use CPLStrtod. Here the parsing is done 'at hand' and should be fast. I'm including a small test program.

The problem with date should probably be addressed at a higher level in the GML driver itself.

by Even Rouault, 16 years ago

by Even Rouault, 16 years ago

Attachment: cpl_test_get_value.c added

Small test program for CPLGetValueType

comment:3 by Even Rouault, 16 years ago

Cc: mlostkot Even Rouault added

comment:4 by warmerdam, 16 years ago

Cc: warmerdam added
Owner: changed from warmerdam to Even Rouault

Even,

Please feel free to go ahead with your suggested fix. But it seems a bit substantial for 1.5.1, so I think we should just keep it in trunk.

comment:5 by warmerdam, 16 years ago

Component: defaultOGR_SF
Keywords: gml added

comment:6 by Even Rouault, 16 years ago

Milestone: 1.6.0

Patch commited in trunk in r13509

comment:7 by Even Rouault, 16 years ago

Resolution: fixed
Status: newclosed

And test added to autotest/cpp/test_cpl.cpp

Note: See TracTickets for help on using tickets.