gdaladdo not producing correct overviews for greyscale+alpha image

[esok]

In gdal 1.6.1, gdaladdo is not producing the correct overviews for an image I'm working on. It works correctly with gdal 1.5.4. The image is very large and can be found at ​http://www.map.nasa.gov/esok/200906191445-goes12-fulldisk-ir4.tiff in case it cannot be attached to this ticket. I then try to run:

gdaladdo -r average 200906191445-goes12-fulldisk-ir4.tiff 2 4 8 16

Symptoms: The image appears blank at some resolutions, notably 512x512 and 256x256, where the greyscale depth goes from 8 to 1. gdal 1.5.4 works correctly and the greyscale depth remains at 8 at the smaller sizes. I used ImageMagick's "identify -verbose" utility to determine bit depths for the overview images.

[Even Rouault]

Yes, regression due to mask bands being taking into account for overview computation in GDAL 1.6, which causes problems with cascaded overview optimization.

Fixed in trunk (r17266) and branches/1.6 (r17267) by disabling cascaded optimization in those cases. Test added in r17268.

Workaround with GDAL 1.6.0 and 1.6.1 : 1) if you can live with neirest neighbour resampling, that would avoid the bug 2) if you really want average resampling, you'll have to build the overview levels one at a time, and in the following order :

gdaladdo -r average 200906191445-goes12-fulldisk-ir4.tiff 2
gdaladdo -r average 200906191445-goes12-fulldisk-ir4.tiff 4
gdaladdo -r average 200906191445-goes12-fulldisk-ir4.tiff 8
gdaladdo -r average 200906191445-goes12-fulldisk-ir4.tiff 16

[Even Rouault]

Frank, you might be interested in backporting that fix in 1.6-esri.

[warmerdam]

Changes ported into 1.6-esri branch (r17274).

Thanks!