Opened 13 years ago

Closed 12 years ago

Last modified 12 years ago

#3797 closed defect (fixed)

netCDF driver ignores the "scale_factor"

Reported by: jluis Owned by: etourigny
Priority: normal Milestone:
Component: GDAL_Raster Version: svn-trunk
Severity: normal Keywords: netcdf
Cc: pds, Kyle Shannon

Description

The scale factor in netcdf files is ignored. I'll use a GMT generated file to show this.

  • Generate one dumb file

grdmath -R-10/10/-10/10 -I1 X Y MUL = lixo.grd

  • Change the scale_factor from its default value of 1 to 0.01

grdedit lixo.grd -D=/=/=/0.01/=/=/=/=

  • Convert to tiff

gdal_translate lixo.grd lixo.tiff

Ask GMT & GDAL about data min/max

grdinfo lixo.grd <...> lixo.grd: x_min: -10 x_max: 10 x_inc: 1 name: x nx: 21 lixo.grd: y_min: -10 y_max: 10 y_inc: 1 name: y ny: 21 lixo.grd: z_min: -1 z_max: 1 name: z lixo.grd: scale_factor: 0.01 add_offset: 0

gdalinfo lixo.tiff -mm <...> Band 1 Block=21x21 Type=Float32, ColorInterp=Gray

Computed Min/Max=-100.000,100.000

NoData Value=nan

What happens is that while converting the scale_factor was not applied.

Attachments (1)

lixo.grd (2.6 KB ) - added by jluis 13 years ago.

Download all attachments as: .zip

Change History (11)

by jluis, 13 years ago

Attachment: lixo.grd added

comment:1 by Even Rouault, 13 years ago

Cc: Kyle Shannon added

comment:2 by Kyle Shannon, 13 years ago

Milestone: 1.8.0
Owner: changed from warmerdam to Kyle Shannon
Status: newassigned

I should be able to get to this this week.

comment:3 by Kyle Shannon, 13 years ago

I have added Set/Get functions for Offset and Scale in the netcdf driver. Ticket #1660 was submitted some time ago and was in reference to this issue too. As Frank mentioned in that ticket:

Note that GDALRasterBand has methods to get the offset and scale. The normal GDAL practice would be to return them via those methods - not to apply them on the fly. Then it is up to the caller to do so if they wish.

I agree with that, in that GDAL is allowing access to the data. This avoids data type issues. In the case above, gdalinfo is the caller. I will commit this after some testing. jluis, do you have any other datasets that are scaled i could use for testing? if not, i will build a few. I will also add a test to autotest.

kss

comment:4 by Kyle Shannon, 13 years ago

Resolution: fixed
Status: assignedclosed

I am going to close this and call it fixed. There may be another issue, but I think it deserves another ticket. Scale and Offset implemented in trunk r20997

comment:5 by etourigny, 12 years ago

Cc: pds added; Kyle Shannon removed
Milestone: 1.8.0
Resolution: fixed
Status: closedreopened

I'm reopening this ticket, because the resolution is not satisfactory.

The result of the discussion at http://www.osgeo.org/pipermail/gdal-dev/2010-October/026544.html seems to be that gdalinfo shouldn't be modified to account for Offset and Scale.

However, this begs the question: what is the use of these RasterBand variables if nothing uses them? Allegedly, one can call gdal_translate -unscale but that is somewhat inconvenient. Further, other apps like QGis do not necessarily support Offset and Scale.

Perhaps a better solution in the netcdf driver is to use internal offset/scale values, and calculate the unscaled valeus in RasterIO() (as suggested in bug #1660). Downside of this, is that code calling RasterIO will probably be expecting Integer (scaled data).

An other option is to unscale the data when creating the file, but this defeats the purpose of scaled data.

comment:6 by etourigny, 12 years ago

Owner: changed from Kyle Shannon to etourigny
Status: reopenednew

comment:7 by etourigny, 12 years ago

Perhaps I do not understand the use and meaning of GDAL's Offset and Scale variables. In netcdf they are intended to be transparent to the user, whereas in GDAL it is up to the user to manage it. Very different from compression packing (i.e. GTIFF DEFLATE) which is transparent at least in reading.

comment:8 by Kyle Shannon, 12 years ago

Cc: Kyle Shannon added

Etienne, I believe that GDAL is for low level access of files, not analysis. It does matter what the unscaled values are in that regard. If the user wants to store 8 bit values and specify scale and offset, they can. If a user of the library wants to unscale values they can. gdalinfo aren't the only apps that use gdal. Unscaling values should be left to the user, imho.

kss

comment:9 by etourigny, 12 years ago

Resolution: fixed
Status: newclosed

Kyle,

Thanks for your input.

I understand your arguments, however it would be much easier if GDAL (gdalinfo) allowed to get the unscaled values for a rapid analysis, like other command-line tools such as nco and cdo. However, I understand that the GDAL philosophy is to keep it as low-level as possible, and not mess with higher-level details.

In light of this I will close re-close this bug, submit a report to QGIS and consider adding an option to unscale in the netcdf driver (-co or config).

comment:10 by etourigny, 12 years ago

Keywords: netcdf added
Note: See TracTickets for help on using tickets.