Opened 12 years ago

Closed 12 years ago

Last modified 12 years ago

#4614 closed defect (fixed)

Incorrect Rounding from OGRFormatDouble

Reported by: jpalmer Owned by: Even Rouault
Priority: high Milestone: 1.9.1
Component: OGR_SF Version: unspecified
Severity: normal Keywords:
Cc: Robert Coup

Description (last modified by jpalmer)

When exporting a geometry with a vertex value of 169.600374, OGRGeometry.ExportToWkt() gives an incorrect rounded value of 169.60037.

Tracking this down it relates to the logic implemented within OGRFormatDouble() pretty-printing rounding when dealing with trailing 99999s:

In the default case where nPrecision is set to 15 the float value is incorrectly truncated to 5 DP.

Change History (3)

comment:1 by jpalmer, 12 years ago

Description: modified (diff)

comment:2 by Even Rouault, 12 years ago

Milestone: 1.9.1
Resolution: fixed
Status: newclosed
Version: svn-trunkunspecified

Fixed in trunk (r24239) and in branches/1.9 (r24240). Hopefully the new logic is a bit less arbitrary than before, but this code is really about heuristics...

comment:3 by jpalmer, 12 years ago

Thanks for the fast fix!

Note: See TracTickets for help on using tickets.