Opened 13 years ago

Closed 13 years ago

#4183 closed enhancement (fixed)

[PATCH] PamFindMatchingHistogram - fix floating-point comparison

Reported by: etiennesky Owned by: Even Rouault
Priority: normal Milestone: 1.9.0
Component: GDAL_Raster Version: 1.8.1
Severity: normal Keywords:
Cc:

Description

The algorithm used for finding matching histogram stats in .aux.xml files uses a simple floating point comparison, which fails in some cases.

The consequence is that histogram information stored in the aux file is not used in subsequent calls, the information must be re-calculated each time that it is requested, and the aux file is polluted with duplicate data.

A preferred method is to compare the difference in floating-point numbers against a given EPSILON value. 

#include <float.h>
#ifndef DBL_IS_EQUAL
#define DBL_IS_EQUAL(a, b) (fabs((a)-(b)) < FLT_EPSILON)
#endif

...

! DBL_IS_EQUAL( atof(CPLGetXMLValue( psXMLHist, "HistMin", "0")), dfMin)

Attached a patch with the proposed fix.

Attachments (1)

patch-gdal-dbl_is_equal.txt​ (1.3 KB ) - added by etiennesky 13 years ago.

Download all attachments as: .zip

Change History (4)

by etiennesky, 13 years ago

Attachment: patch-gdal-dbl_is_equal.txt​ added

comment:1 by etiennesky, 13 years ago

Summary: PamFindMatchingHistogram - fix floating-point comparison → [PATCH] PamFindMatchingHistogram - fix floating-point comparison

comment:2 by Even Rouault, 13 years ago

Owner: changed from warmerdam to Even Rouault

comment:3 by Even Rouault, 13 years ago

Milestone: → 1.9.0
Resolution: → fixed
Status: new → closed

r22857 /trunk/gdal/ (6 files in 4 dirs): PamFindMatchingHistogram - fix floating-point comparison; rename EQUAL_TO_NODATA() macro into ARE_REAL_EQUAL() (and make it work when 2nd operand is strictly 0) because it is usefull in other contexts than nodata comparison (#4183)

r22858 /trunk/gdal/gcore/gdalpamrasterband.cpp: commit forgotten file (#4183)

Note: See TracTickets for help on using tickets.