Opened 13 years ago

Closed 13 years ago

Last modified 13 years ago

#3573 closed defect (fixed)

gdalinfo -stats and numeric imprecision on dataset with nodata

Reported by: maphew Owned by: Even Rouault
Priority: normal Milestone: 1.7.3
Component: Utilities Version:
Severity: normal Keywords: gdalinfo
Cc: Kyle Shannon, jjr8


When I run gdalinfo -stats on a 32bit 1-band tiff (elevation model) it renders the image unviewable in Arcmap 9.3.1sp1. Arccatalog & Arcmap report all the usual info as for as datatype presence of pyramids and so on, it just draws as white. There is no error message.

Before calculating statistics Arcmap reports high/low values in the form of:

    High : 3.40282e+038
    Low : 1.17549e-038

and after gdalinfo -stats it becomes:

    High : 1952.5
    Low : -3.40282e+038

After playing with Arcmap symbology and using contrast stretch, std devations, etc. The Low changes to:

    Low : -340282346638529990000000000000000000000.000000

Arccatalog reports NoData: -3.4028235e+038

The raster can be made to display by instructing Arcmap to use "Stretched>Min/Max>Edit-High-Low" and change the bottom end to the real minimum value, or at least 0. This is not a real solution as it's only good for that particular Arcmap session. Deleting the "foobar.tif.aux.xml"works better.

Using ArcGIS to calulate statistics on the same file after deleting .aux.xml reveals the true min/max statistics:

    High : 580.340637
    Low : 213.000000

I'm using GDAL 1.7.0dev, released 2008/11/26 on Windows7 x64 via OSGeo4w.

Attachments (1)

stats-no-draw.7z (521.9 KB ) - added by maphew 13 years ago.
data set with original elevation model, after gdal stats and after arcgis stats

Download all attachments as: .zip

Change History (6)

by maphew, 13 years ago

Attachment: stats-no-draw.7z added

data set with original elevation model, after gdal stats and after arcgis stats

comment:1 by Kyle Shannon, 13 years ago

Cc: Kyle Shannon added

comment:2 by Even Rouault, 13 years ago

Owner: changed from warmerdam to Even Rouault
Summary: gdalinfo -stats incompatible with ArcMap 9.3.1gdalinfo -stats and numeric imprecision on dataset with nodata

comment:3 by jjr8, 13 years ago

Cc: jjr8 added

comment:4 by Even Rouault, 13 years ago

Milestone: 1.7.3
Resolution: fixed
Status: newclosed
Version: 1.7.0

The reason is that the nodata value wasn't ignored when computing the stats. Nodata value is stored in GeoTIFF files as a text value, but the pixel values are stored as IEEE floats. So strict comparison can fail. The fix is to test that the relative difference between nodata and pixel values is small (1e-10), but not exactly 0.

Fixed in trunk (r19669) and branches/1.7 (r19671). Test added in r19670

After fix, gdalinfo -stats small/orig/dem.tif returns :

Band 1 Block=493x4 Type=Float32, ColorInterp=Gray
  Minimum=213.000, Maximum=578.979, Mean=355.119, StdDev=84.672
  NoData Value=-3.40282346638528898e+38

comment:5 by maphew, 13 years ago

wow, that was fast. Thanks Evan :)

Note: See TracTickets for help on using tickets.