Memory driver bug no data value (-9999) stored 0 on memory file

[aly]

please see here ​http://gis.stackexchange.com/questions/183771/gdal-memory-driver-no-data-value-9999-converted-to-0

[Even Rouault]

Please provide a fully reproducible way of hitting this. The following works fine for me

from osgeo import gdal, gdal_array
import numpy
ds = gdal.GetDriverByName('MEM').Create('', 1, 1, 1, gdal.GDT_Float32)
ds.GetRasterBand(1).SetNoDataValue(-9999.0)
ar = numpy.array([[-9999.0]], dtype = numpy.float32)
ds.GetRasterBand(1).WriteArray(ar)
print(ds.GetRasterBand(1).ReadAsArray())

outputs

[[-9999.]]

[aly]

In my case, I accidently created a large array shape=(3,3), while I have (1,1). However, excess elements should be set to no data since I set nodata as -9999.0. Because this is a temperature raster array, 0 has a meaning of 0 Celcius while no data has no meaning.

try this.

from osgeo import gdal, gdal_array
import numpy
ds = gdal.GetDriverByName('MEM').Create('', 3, 3, 1, gdal.GDT_Float32)
ds.GetRasterBand(1).SetNoDataValue(-9999.0)
ar = numpy.array([[-9999.0]], dtype = numpy.float32)
ds.GetRasterBand(1).WriteArray(ar)
print(ds.GetRasterBand(1).ReadAsArray())

[[-9999.     0.     0.]
 [    0.     0.     0.]
 [    0.     0.     0.]]

[Even Rouault]

hum ok, well this is a bit expected behaviour. SetNoDataValue() generally has no effect on the pixel values. In the case of the MEM driver, the buffers are initialized to 0 on creation, and SetNoDataValue() does not do anything. My advice would be to call ds.GetRasterBand(1).Fill(-9999) in addition to SetNoDataValue()

Some drivers like GTiff or HFA would behave more like what you expect, but this is because they have a concept of initialized/present blocks, which the MEM driver doesn't have. So when they load a block not initialized yet, they return values set to the nodata value (or 0 if it isn't set)

For the MEM driver, we could perhaps have a boolean flag meaning "has the raster been explictly written yet ?" and if false, then SetNoDataValue() would do thet Fill() implictly. But I'm not completely convinced we should do that. The call to Fill() remains a simple solution.

I've just improved the doc : trunk r33675 "SetNoDataValue(): better document potential pitfalls (#6404)"

[aly]

This also causes problem in reproject function (maybe more functions), as well when you write the array on memory file. However, when you write the array to geotiff, it is good. see here, ​http://gis.stackexchange.com/questions/158503/9999-no-data-value-becomes-0-when-writing-array-to-gdal-memory-file.

For this reason, the buffers on memory file must be initialized to no data value as in geotiff only if no data value is set by the user. You are only right when I don't set no data value explicitly. 0 has a meaning so in my case, when I have temperature data, you initialize the raster with 0 Celcius instead of no data value.

Besides, I can not fill the array on memory file with -9999 because I don't know the output array shape before reprojection. It is being calculated by reproject function.

[aly]

Replying to aly:

please see here ​http://gis.stackexchange.com/questions/183771/gdal-memory-driver-no-data-value-9999-converted-to-0

see here, ​​http://gis.stackexchange.com/questions/158503/9999-no-data-value-becomes-0-when-writing-array-to-gdal-memory-file.

[Even Rouault]

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub. If you believe this ticket is still valid, you may file it to ​https://github.com/OSGeo/gdal/issues if it is not already reported there.