#4614 closed defect (fixed)
Incorrect Rounding from OGRFormatDouble
| Reported by: | jpalmer | Owned by: | Even Rouault |
|---|---|---|---|
| Priority: | high | Milestone: | 1.9.1 |
| Component: | OGR_SF | Version: | unspecified |
| Severity: | normal | Keywords: | |
| Cc: | Robert Coup |
Description (last modified by )
When exporting a geometry with a vertex value of 169.600374, OGRGeometry.ExportToWkt() gives an incorrect rounded value of 169.60037.
Tracking this down it relates to the logic implemented within OGRFormatDouble() pretty-printing rounding when dealing with trailing 99999s:
http://trac.osgeo.org/gdal/browser/trunk/gdal/ogr/ogrutils.cpp#L141
In the default case where nPrecision is set to 15 the float value is incorrectly truncated to 5 DP.
Change History (3)
comment:1 by , 12 years ago
| Description: | modified (diff) |
|---|
comment:2 by , 12 years ago
| Milestone: | → 1.9.1 |
|---|---|
| Resolution: | → fixed |
| Status: | new → closed |
| Version: | svn-trunk → unspecified |
Note:
See TracTickets
for help on using tickets.
Fixed in trunk (r24239) and in branches/1.9 (r24240). Hopefully the new logic is a bit less arbitrary than before, but this code is really about heuristics...