Opened 11 years ago

Closed 4 years ago

#4398 closed defect (wontfix)

OGR shapefile driver, field numerical scientific notation

Reported by: rsbivand Owned by: warmerdam
Priority: normal Milestone: closed_because_of_github_migration
Component: default Version: unspecified
Severity: normal Keywords: OGR Shapefile write
Cc:

Description

Recent ESRI shapefiles apparently write very small floating point numbers in "e" format, which can be read but not be written with shapelib or the ESRI shapefile driver. A possible resolution may be to edit lines 1347-8 in:

http://trac.osgeo.org/gdal/browser/trunk/gdal/ogr/ogrsf_frmts/shape/dbfopen.c

sprintf( szFormat, "%%%d.%df",

nWidth, psDBF->panFieldDecimals[iField] );

to accommodate a possible scientific notation. The attached shapefile is seeded with 9.87654e-24 in reg_flt, and reads fine, but when writing the same file out, the DBF file gets "0.000000000000000" rather than the ESRI DBF, which has "9.87654e-024". I realise that ESRI are probably stretching what may be put into a DBF, but if they are doing this, it might be reasonable to do the same.

Attachments (3)

sn.zip (560 bytes ) - added by rsbivand 11 years ago.
examples1.zip (192.7 KB ) - added by pet221 11 years ago.
6 shapefiles & R script
examples2.zip (793.0 KB ) - added by pet221 11 years ago.

Download all attachments as: .zip

Change History (8)

by rsbivand, 11 years ago

Attachment: sn.zip added

comment:1 by rsbivand, 11 years ago

The problem can be seen by:

cat e_notation.dbf

ogr2ogr -f "ESRI Shapefile" t2.shp e_notation.shp cat t2.dbf

comment:2 by rsbivand, 11 years ago

In ogr/ogrsf_frmts/shape/dbfopen.c

if (abs(*((double *) pValue)) > 1e-6)

sprintf( szFormat, "%%%d.%df",

nWidth, psDBF->panFieldDecimals[iField] );

else sprintf(szFormat, "%%.%de", nWidth - 8);

appears to provide a remedy, but there are two arbitrary constants - the break point for switching formats and the width provided for the scientific notation. It works in ogr2ogr and called within R, both on the sample file created in ArcGIS and on R-generated files.

by pet221, 11 years ago

Attachment: examples1.zip added

6 shapefiles & R script

by pet221, 11 years ago

Attachment: examples2.zip added

comment:3 by pet221, 11 years ago

I've attached two zipped directories (examples1 and examples2) containing a script (exampleDBF.r) with R code that further demonstrates the issue, as well as six small example shapefiles created in ArcGIS version 9.3.1. We are using a locally modified shapelib (see previous post) and it seems to remedy the issue.

comment:4 by Jukka Rahkonen, 8 years ago

Shapefile specification says:

"Any floating point number smaller than [1.0e-38] is considered by a shapefile reader to represent a "no data" value."

Thus 9.87654e-24 is well within the range that should be supported.

comment:5 by Even Rouault, 4 years ago

Milestone: closed_because_of_github_migration
Resolution: wontfix
Status: newclosed

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub. If you believe this ticket is still valid, you may file it to https://github.com/OSGeo/gdal/issues if it is not already reported there.

Note: See TracTickets for help on using tickets.