Opened 3 years ago

Closed 8 months ago

#6804 closed enhancement (wontfix)

Shapefile/DBF: consider how to deal with non decimal point separator in numeric fields

Reported by: mbernasocchi Owned by: warmerdam
Priority: normal Milestone: closed_because_of_github_migration
Component: default Version: 1.11.3
Severity: normal Keywords:
Cc:

Description

I have a data set created using LC_NUMERIC=it_IT.UTF-8 which is a comma. QGIS displays the data without the after comma part.

Looking up the coode at [1, 2] looks like SetField? uses CPLStrtod which does not take locale into account. Is this supposed to be so? (sorry, I'm a python dev so I might have misjudged some of the pointery stuff :) ) I've also looked up in gdal 2.1 and the code stays similar [3]

[1] https://github.com/OSGeo/gdal/blob/tags/1.11.3/gdal/ogr/ogrfeature.cpp#L2360

[2] https://github.com/OSGeo/gdal/blob/tags/1.11.3/gdal/port/cpl_strtod.cpp#L321

[3] https://github.com/OSGeo/gdal/blob/tags/2.1.3/gdal/ogr/ogrfeature.cpp#L3096

Attachments (1)

data_with_locale.zip (147.6 KB) - added by mbernasocchi 3 years ago.

Download all attachments as: .zip

Change History (6)

Changed 3 years ago by mbernasocchi

Attachment: data_with_locale.zip added

comment:1 Changed 3 years ago by Jukka Rahkonen

Perhaps related to QGIS issue https://hub.qgis.org/issues/8332.

I live in Finland which is a comma-land as well but never hit this issue. I guess I have been lucky and never seen .dbf file written with comma as a decimal separator before this.

comment:2 Changed 3 years ago by Even Rouault

A DBF with a decimal separator other than point is invalid IMHO. We could be nice and try detecting this though, but that could cause consistency issues since on update we'd use use decimal point

comment:3 Changed 3 years ago by mbernasocchi

Salut Even, I agree that comma in the dbf is bad, but I also think that truncating a value should maybe be an error and not only a warning?

Also setting

export OGR_SETFIELD_NUMERIC_WARNING=YES
or

ogr2ogr --config OGR_SETFIELD_NUMERIC_WARNING YES -f "ESRI Shapefile" /tmp/test2.shp custom_operator.shp

does not give any warnings on GDAL 1 (we tested 1.10.1 on ubuntu 14.04, 1.11.3 on ubuntu 16.04, 1.11.4 on centos7)

GDAL 2 gives the warning by default (tested on 2.1.2 fedora and 2.1.1 Ubuntu 16.10)

Lastly, I couldn't find the decimal separator in the dbf specs (http://www.dbase.com/KnowledgeBase/int/db7_file_fmt.htm)

comment:4 Changed 3 years ago by Even Rouault

Summary: OGRFeature::SetField ignores LC_NUMERIC?Shapefile/DBF: consider how to deal with non decimal point separator in numeric fields
Type: defectenhancement

comment:5 Changed 8 months ago by Even Rouault

Milestone: closed_because_of_github_migration
Resolution: wontfix
Status: newclosed

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub?. If you believe this ticket is still valid, you may file it to https://github.com/OSGeo/gdal/issues if it is not already reported there.

Note: See TracTickets for help on using tickets.