Opened 12 years ago

Closed 12 years ago

Last modified 12 years ago

#2968 closed defect (fixed)

OGR GPX driver loses time field in shapefile

Reported by: hamish Owned by: warmerdam
Priority: normal Milestone: 1.7.0
Component: OGR_SF Version: 1.5.2
Severity: normal Keywords: gpx
Cc:

Description

Hi,

thanks for looking at the two GPX issues I reported yesterday. Here's another one for you:

If using ogr2ogr to convert a GPX file containing a track into a shapefile the 'time' Field gets broken and you end up with no time data in the shapefile's DBF.

for example:

$ ogr2ogr  -f "ESRI Shapefile" test_trk_shp test_trk.gpx
Warning 6: Field time create as date field, though DateTime requested.

Warning 6: Field time create as date field, though DateTime requested.

Warning 6: Field time create as date field, though DateTime requested.
$ dbview -e test_trk_shp/track_points.dbf | head
Field Name      Type    Length  Decimal Pos
track fid         N        11       0
track seg         N        11       0
track seg         N        11       0
ele               N        24      15
time              D         8       0
magvar            N        24      15
geoidheigh        N        24      15
name              C        80       0
cmt               C        80       0
...
# DBF contains records 4000 data points, half with no timestamp (doh!)
$ dbview track_points.dbf | grep '^time' | uniq
time       : 00000000
time       : 

i.e. when the <time> GPX element is missing from the <trkpt> "00000000" is written. when a <time> value is in the <trkpt> record it is written empty.

good times look like:

<time>2009-04-28T06:31:53Z</time>

and for many needs recording time to the nearest second is needed. e.g. we are adding spatial data to recording equipment logs which record timestamps but are not directly connected to the GPS.

if DateTime? is not available could it write to (e.g.) varchar(255) instead?

thanks, Hamish

Attachments (1)

test_trk2.gpx (4.8 KB) - added by hamish 12 years ago.
example gpx file with partial missing timestamps

Download all attachments as: .zip

Change History (7)

comment:1 Changed 12 years ago by Even Rouault

I'm a bit confused of which <time> element you are speaking about. Could you attach a small GPX file demonstrating the issue ?

Changed 12 years ago by hamish

Attachment: test_trk2.gpx added

example gpx file with partial missing timestamps

comment:2 in reply to:  1 Changed 12 years ago by hamish

Replying to rouault:

I'm a bit confused of which <time> element you are speaking about. Could you attach a small GPX file demonstrating the issue ?

ok, attached. the resulting track_points.shp's .dbf is losing the <time> field in this:

<trkpt lat="-46.035737442" lon="170.171801448">
  <ele>9.750610</ele>
  <time>2009-04-27T02:38:41Z</time>
  <extensions>
    <gpxx:Depth>8.128929</gpxx:Depth>
    <gpxx:Temperature>12.739347</gpxx:Temperature>
  </extensions>
</trkpt>

thanks, Hamish

comment:3 Changed 12 years ago by Even Rouault

Milestone: 1.6.11.7.0
Resolution: fixed
Status: newclosed

OK, yes, this was an issue in GDAL 1.5.2

  • From GDAL 1.5.4, field of type DateTime? are written in shapefiles as fields of type Date (See http://trac.osgeo.org/gdal/ticket/2474). But of course, this keep only the day part and loses the hour:minute:second part of a datetime field
  • From GDAL 1.6.0, you can use the OGR SQL CAST operator to convert any field to another type. For example :
    ogr2ogr out source.gpx -sql 'select ele, cast (time as character(32)) from track_points'
    
  • In trunk (GDAL 1.7.0dev r16894), I've introduced a new option '-fieldTypeToString' for ogr2ogr that is a convenient way of casting all field of given type(s) to fields of type string in the destination layer. For example :
    ogr2ogr out source.gpx -fieldTypeToString DateTime
    
  • In branches/1.6 (r16895), I've just backported the core fix done in OGRFeature::SetFrom?(), so that datetime fields can be written as strings

comment:4 Changed 12 years ago by Even Rouault

Added (in r16897) doc for the issue discussed in that ticket

comment:5 in reply to:  3 Changed 12 years ago by hamish

Replying to rouault:

  • In branches/1.6 (r16895), I've just backported the core fix done in OGRFeature::SetFrom?(), so that datetime fields can be written as strings

thanks. For 1.7dev it would seem to me that writing the time as a string should be the default for shapefiles (lacking DateTime? support), rather than to be intentionally 50% lossy due to the partial name overlap of the types. the loss of precision is severe. or another way to think about it, the nearest neighbor type-name is not always the best choice.

cheers, Hamish

comment:6 Changed 12 years ago by Even Rouault

It depends on the use cases. You may want that the column is of type Date. Anyway, now you have full control on how to output it in the shapefile. And I'd note that if you try to write a DateTime? field, the shapefile driver issues a warning : "Warning 6: Field time create as date field, though DateTime? requested."

Note: See TracTickets for help on using tickets.