Just Generated Pixel Interleaved TIFF Overviews Appear as Zeros

[warmerdam]

When internal overviews are built in a GeoTIFF file that is pixel interleaved, the overviews appear to be all zero valued. The same is not true of a band interleaved file. If the dataset is closed and reopened the imagery is fine. Tracking it down, IsBlockAvailable() is returning FALSE when it presumably should be TRUE, but I'm not sure why. The problem can be demonstrated with this program and the attached file.

#include "gdal_priv.h"

int main()

{
    GDALDataset *poDS;

    GDALAllRegister();

    system( "cp mfloat_base.tif mfloat32.tif" );
    poDS = (GDALDataset *) GDALOpen( "mfloat32.tif", GA_Update );

    int anOverviews[2] = { 2, 4 };					
    GDALRasterBand *poOver;

    poDS->BuildOverviews( "NEAREST", 2, anOverviews, 0, NULL, 
                          GDALDummyProgress, NULL );
    
    poOver = poDS->GetRasterBand(3)->GetOverview(0);

    double dfMax;
    poOver->ComputeStatistics( FALSE, NULL, &dfMax, NULL, NULL, 
                               NULL, NULL );

    printf( "Max = %g\n", dfMax );

    delete poDS;
}

[warmerdam]

input file for example program.

[warmerdam]

Temporarily forced gdalautotest/gcore/tiff_ovr.py (tiff_ovr_1) test to use interleave=band to work around this in test suite (r11891). Please remove this setting when bug fixed.

[warmerdam]

This may be related to #1758.

[Even Rouault]

I think I've understood what happens here. I've simplified a bit your test case to reduce to just 1 overview and 2 bands. The following analysis is based on this simplified test case.

When generating the overview, there are finally 2 calls to GTiffRasterBand::IWriteBlock (one for each band). In the pixel interleaved case (for optimization purpose apparently ?), a block is not written right away. Instead the flag poGDS->bLoadedBlockDirty = TRUE; is set. If one writes to another block, the method GTiffDataset::LoadBlockBuf will flush the previous block on the disk through the call to GTiffDataSet::FlushBlockBuf (it will also be flushed if GTiffDataset::FlushCache is called). Here, none of those 2 cases happen. When computing the stat, we issue a call to GTiffRasterBand::IReadBlock and this method supposes that the data is flushed... which is not the case. Bang ! That accounts for IsBlockAvailable returning 0.

My proposal for the fix is to have an explicit call to poGDS->FlushBlockBuf() just after the call to poGDS->SetDirectory() at the beginning of GTiffRasterBand::IReadBlock, GTiffRGBABand::IReadBlock, GTiffBitmapBand::IReadBlock and GTiffOddBitsBand::IReadBlock, so that previously cached writes are flushed before reading. See attached patch.

The fix works for this test case and IMHO shouldn't cause much harm for other cases.

The bad news is that it doesn't improve resolution of bug #1758.

[warmerdam]

Even,

Bingo! You are exactly right. I patched it slightly different, modifying SetDirectory() to flush a loaded block before changing directory and some other steps to avoid recursion but roughly the same change.

Patched in trunk (r11999), 1.4 (r12001).

I also restored tiff_ovr.py to check for this issue (r12000).