.dbf files; inconsistencies with date formats

[j03lar50n]

Inconsistencies in date formats within .dbf's of shapefiles have been noticed when loading a particular .shp in QGIS, OpenJump, MapWindow and OGR.

In particular the sample .dbf file has a installe_1 date field for which the first record is reported by OGR as 0000/00/04 while the actual text in the file (as reported by shapelib's dbfdump) is "04/25/1989".

[j03lar50n]

.dbf file for Ticket #2746

[warmerdam]

Roni,

Please treat this as your top priority bug report to work on. You will need to review the literature on dbf file date formatting to figure out if there is some systematic way to know date formats in .dbf formats. The OGR Shapefile driver uses shapelib for dbf access (​http://shapelib.maptools.org) (also maintained by me), but the date support is quite new. ​http://www.clicketyclick.dk/databases/xbase/format/ may be a useful resource on dbf formats.

We must take care not to break other dbf files with dates while fixing this ticket. Note that this file is properly supported by some other applications (mapwindow, and excel) which presumably use some microsoft supplied dbf parser.

[ashrafroni]

I used jump-src-1.1.2 and modified the source to load only water_main_dist.dbf and i got unparseable date. I went through the source code.

And got the date format in water_main_dist.dbf is Month.date.Year but the standard format jump expect is Year.Month.Date. In Excel shows it as String value only.So it can show the right format. In the dbf file the date is in text(String) format. I also want to know which version of open jump is used.

I will check the OGR and let you know details.

[j03lar50n]

Replying to ashrafroni:

I also want to know which version of open jump is used.

I cannot get the shapefile to load in OpenJUMP version 1.2F .

Output Window msg:

1 Problem Loading water_main_dist. See View / Log For Stack Traces For input string: "'" (Number Format Exception)

Shapefile loads and date field attributes are readable in OpenEV.

[ashrafroni]

The ogrinfo expected the water_main_dist.dbf date field(installe_1) should be in YYYYMMDD fromat. For reading the date information from the database it used the following line of code:

int nFullDate = DBFReadIntegerAttribute( hDBF, iShape, iField );

But in the database file it is in MM/DD/YYYY format, so when it executes the upper line it showed wrong value. Now both YYYYMMDD and MM/DD/YYYY are being supported.

Please check the attachment.

[warmerdam]

Re-open pending patch review and application.

[warmerdam]

Roni,

Since you started working on this (in fact just in the last couple days) the shape2ogr.cpp file was modified for other issues. Please "svn update" to get those changes and then attach a patch generated with "svn diff shape2ogr.cpp".

Thanks,

[Even Rouault]

Roni,

instead of using strcpn and strtok, you could use CSLTokenizeString()/CSLCount()/CSLDestroy() from the GDAL port library (/port subdirectory). See ​http://gdal.org/cpl__string_8h.html

[ashrafroni]

Thanks rouault, Thats a good idea. I will change this immediately and send the svn diff patch soon.

Regards Roni

[ashrafroni]

I changed the string function using cpl_string.

[warmerdam]

Reopening pending application of the patch(es).

[warmerdam]

I have added a slight variation of the patch that more carefully confirms things are in MM/DD/YYYY format by checking at which offsets the slash is found.

Fix in trunk (r16200), and 1.6 branch (r16201) with a new test suite addition (r16199).

Thanks Roni!

[Even Rouault]

Just a minor note : if someone updates date fields in such a .dbf, we would end up mixing 2 different date formats.