GDAL/GeoTIFF/GRASS color table bug

[Markus Neteler]

Hi Frank,

unfortunately we found another problem: When exporting GRASS data with
gdal_translate to GeoTIFF (float map), the colors are found but later
on detected as grey scale.

What we did:

gdal_translate -of GTIFF /ssi0/ssi/neteler/grassdata/pat/neteler/cellhd/muell
out.tif
 RGB 0 0 255
 RGB 57 151 105
 RGB 58 151 105
 RGB 58 152 105
 RGB 59 152 105
[...]
 RGB 225 218 119
 RGB 225 217 119
Size is 492, 411
0...10...20...30...40...50...60...70...80...90...100 - done.

Note: the RGB output I added to see if colors were found (added in
function GRASSRasterBand around line 216.

gdalinfo out.tif
Driver: GTiff/GeoTIFF
Size is 492, 411
Coordinate System is `'
Origin = (1666520.000000,5126890.000000)
Pixel Size = (10.000000,-10.000000)
Corner Coordinates:
Upper Left  ( 1666520.000, 5126890.000)
Lower Left  ( 1666520.000, 5122780.000)
Upper Right ( 1671440.000, 5126890.000)
Lower Right ( 1671440.000, 5122780.000)
Center      ( 1668980.000, 5124835.000)
Band 1 Block=492x4 Type=Float32, ColorInterp=Gray
                                 ^^^^^^^^^^^^^^^^^

We tried as well with HDF4 export and looked and the HDF file with
HDFLook, again the map is grey.

Probably 
poCT->SetColorEntry( iColor, &sColor );
is failing?

Thanks in advance,

 Markus

[warmerdam]

Markus,

Could I get a small dataset that demonstrates this?  A LOCATION?

[warmerdam]

On second thought, I see the data is 32bit floating point.  An issue here is
that GeoTIFF only allows color maps for 8bit data.  You should be able to 
scale the data to 8bit using the scaling options in gdal_translate along with
the "-ot Byte" option to force 8bit output.  If you find that this doesn't
work either then get back to me. 

Of course, it would be nice if something in here warned the color table was being
ignored.  I will look into fixing this. 

[neteler@…]

Frank,

thanks for looking into this. I have stored a test location in
ftp.velocet.ca
pub/incoming/testloc2_grass.tar.gz
(please ignore testloc_grass.tar.gz which does not contain an INT map)
which contains 
 - a float DTM with NULL data in the range of
   around 260m to 1120m (can't remember the exact values).
 - a CELL map (INT) with values 0, 1, NULL

Tests:

map 'dtm' (FCELL):
 -the export to 8bit geotiff contains colors but NULL data are coded
  to blue color (value: 0 0 254), should be probably 255 255 255 (?)
  to be white or 0 0 0 black
  [but: export of such DTM to 8bit does not make much sense of course]
 - export to HDF4Image also writes grey scale (without -ot Byte). I guess
   that HDF supports also float color (not sure).
 - export to AAIGrid is fine (NULL is coded properly as 12345)
   [but of course no color table here...]
 - yes, it would be nice if something in here warned the color table was being
   ignored.

map 'muell' (CELL):
 - gdal_translate -of AAIgrid 
/ssi0/ssi/neteler/grassdata/pat/neteler/cellhd/muell out.aai
 delivers a map with 
 NODATA_value      0
      0      0      0      0      0      0      0      0      0      0      0
 [...]
 So the NULL values are found, but the zero's are also written as 0.
 May I recommend to map NULL to -9999 or something better?
 (this is more related to the other bug report which is closed)
- gdal_translate -of Gtiff /ssi0/ssi/neteler/grassdata/pat/neteler/cellhd/muell
out.tif
  -> the map contains two values (instead of three): 0 and 1
     both black
  -> note that this map does not have a color table (I forgot to apply),
     GRASS standard rainbow colors are used inside GRASS:
          0: yellow
          1: red
          NULL: white
     Perhaps, if GDAL doesn't find a color table, it could also use the
     GRASS rainbow table?
- after adding a color table to 'muell'
 r.colors muell col=rules
 0 blue
 1 green
 end
 and export to GTIFF again, the result is in out.tif
           0: blue
           1: green
           NULL: blue
  due to the wrong encoding of NULL (so color transfer works if a GRASS
  color table is present).

Sorry for being so chatty,

 Markus

[warmerdam]

Markus,

Lots of points in this bug report!

The export of 'dtm' to 8bit geotiff is working properly though I agree the
result is unfortunate.  GeoTIFF has no concept of nodata values, and in any
event I think the nodata concept is lost in the process of scaling.  

I believe you are correct that HDF does not support colormaps. 

I have modified geotiff.cpp so that it warns if it does not export a colormap.

On the export of muell to AIGrid.  I can't easiliy map nodata to -9999 because
the source grass muell raster is treated as 8bit by GDAL.  What I have done is
change the grass driver to check the min/max for the raster and to map the
nodata value outside the min/max range if possible for eight bit rasters.  In
this case (muell) it now maps nodata values to 255 so they can be distinguished
from the value values of 0 and 1.   This is also important in the export to
geotiff.

On the idea of default colormaps, I don't think that is wise.  It is perfectly
fine in GDAL to have rasters without colormaps, and I am hesitant to mess with
that. 

Best regards,

[neteler@…]

Frank,

thanks for (again) the quick fixes!

- GTIFF missing color table works
- AIGrid NULL problem solved (the range trick is good)
- no default colormaps is fine of course
- that GeoTIFF has no concept of nodata values, it rather poor
  but certainly not your fault.

Thanks again, we are happy now.

 Markus