Opened 9 years ago

Closed 9 years ago

Last modified 9 years ago

#4614 closed defect (fixed)

Incorrect Rounding from OGRFormatDouble

Reported by: jpalmer Owned by: Even Rouault
Priority: high Milestone: 1.9.1
Component: OGR_SF Version: unspecified
Severity: normal Keywords:
Cc: Robert Coup

Description (last modified by jpalmer)

When exporting a geometry with a vertex value of 169.600374, OGRGeometry.ExportToWkt?() gives an incorrect rounded value of 169.60037.

Tracking this down it relates to the logic implemented within OGRFormatDouble() pretty-printing rounding when dealing with trailing 99999s:

http://trac.osgeo.org/gdal/browser/trunk/gdal/ogr/ogrutils.cpp#L141

In the default case where nPrecision is set to 15 the float value is incorrectly truncated to 5 DP.

Change History (3)

comment:1 Changed 9 years ago by jpalmer

Description: modified (diff)

comment:2 Changed 9 years ago by Even Rouault

Milestone: 1.9.1
Resolution: fixed
Status: newclosed
Version: svn-trunkunspecified

Fixed in trunk (r24239) and in branches/1.9 (r24240). Hopefully the new logic is a bit less arbitrary than before, but this code is really about heuristics...

comment:3 Changed 9 years ago by jpalmer

Thanks for the fast fix!

Note: See TracTickets for help on using tickets.