Opened 8 years ago

Closed 9 months ago

#4398 closed defect (wontfix)

OGR shapefile driver, field numerical scientific notation

Reported by: rsbivand Owned by: warmerdam
Priority: normal Milestone: closed_because_of_github_migration
Component: default Version: unspecified
Severity: normal Keywords: OGR Shapefile write
Cc:

Description

Recent ESRI shapefiles apparently write very small floating point numbers in "e" format, which can be read but not be written with shapelib or the ESRI shapefile driver. A possible resolution may be to edit lines 1347-8 in:

http://trac.osgeo.org/gdal/browser/trunk/gdal/ogr/ogrsf_frmts/shape/dbfopen.c

sprintf( szFormat, "%%%d.%df",

nWidth, psDBF->panFieldDecimals[iField] );

to accommodate a possible scientific notation. The attached shapefile is seeded with 9.87654e-24 in reg_flt, and reads fine, but when writing the same file out, the DBF file gets "0.000000000000000" rather than the ESRI DBF, which has "9.87654e-024". I realise that ESRI are probably stretching what may be put into a DBF, but if they are doing this, it might be reasonable to do the same.

Attachments (3)

sn.zip (560 bytes) - added by rsbivand 8 years ago.
examples1.zip (192.7 KB) - added by pet221 8 years ago.
6 shapefiles & R script
examples2.zip (793.0 KB) - added by pet221 8 years ago.

Download all attachments as: .zip

Change History (8)

Changed 8 years ago by rsbivand

Attachment: sn.zip added

comment:1 Changed 8 years ago by rsbivand

The problem can be seen by:

cat e_notation.dbf

ogr2ogr -f "ESRI Shapefile" t2.shp e_notation.shp cat t2.dbf

comment:2 Changed 8 years ago by rsbivand

In ogr/ogrsf_frmts/shape/dbfopen.c

if (abs(*((double *) pValue)) > 1e-6)

sprintf( szFormat, "%%%d.%df",

nWidth, psDBF->panFieldDecimals[iField] );

else sprintf(szFormat, "%%.%de", nWidth - 8);

appears to provide a remedy, but there are two arbitrary constants - the break point for switching formats and the width provided for the scientific notation. It works in ogr2ogr and called within R, both on the sample file created in ArcGIS and on R-generated files.

Changed 8 years ago by pet221

Attachment: examples1.zip added

6 shapefiles & R script

Changed 8 years ago by pet221

Attachment: examples2.zip added

comment:3 Changed 8 years ago by pet221

I've attached two zipped directories (examples1 and examples2) containing a script (exampleDBF.r) with R code that further demonstrates the issue, as well as six small example shapefiles created in ArcGIS version 9.3.1. We are using a locally modified shapelib (see previous post) and it seems to remedy the issue.

comment:4 Changed 5 years ago by Jukka Rahkonen

Shapefile specification says:

"Any floating point number smaller than [1.0e-38] is considered by a shapefile reader to represent a "no data" value."

Thus 9.87654e-24 is well within the range that should be supported.

comment:5 Changed 9 months ago by Even Rouault

Milestone: closed_because_of_github_migration
Resolution: wontfix
Status: newclosed

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub?. If you believe this ticket is still valid, you may file it to https://github.com/OSGeo/gdal/issues if it is not already reported there.

Note: See TracTickets for help on using tickets.