AIG datasets with colormap

[rprinceley]

In HFARasterBand::HFARasterBand(), we need a way to specify that padfBins should be ignored. This is for compatibility with ArcGIS 9.x which does not map entries to "BinValues?" (when reading .aux).

/* -------------------------------------------------------------------- */
/*      Collect color table if present.                                 */
/* -------------------------------------------------------------------- */
    double    *padfRed, *padfGreen, *padfBlue, *padfAlpha, *padfBins;
    int       nColors;

    if( iOverview == -1
        && HFAGetPCT( hHFA, nBand, &nColors,
                      &padfRed, &padfGreen, &padfBlue, &padfAlpha,
                      &padfBins ) == CE_None
        && nColors > 0 )
    {
        poCT = new GDALColorTable();
        for( int iColor = 0; iColor < nColors; iColor++ )
        {
            GDALColorEntry   sEntry;

            // The following mapping assigns "equal sized" section of 
            // the [0...1] range to each possible output value and avoid
            // rounding issues for the "normal" values generated using n/255.
            // See bug #1732 for some discussion.
            sEntry.c1 = MIN(255,(short) (padfRed[iColor]   * 256));
            sEntry.c2 = MIN(255,(short) (padfGreen[iColor] * 256));
            sEntry.c3 = MIN(255,(short) (padfBlue[iColor]  * 256));
            sEntry.c4 = MIN(255,(short) (padfAlpha[iColor] * 256));
            
            if( padfBins != NULL )
                poCT->SetColorEntry( (int) padfBins[iColor], &sEntry );
            else
                poCT->SetColorEntry( iColor, &sEntry );

The attached dataset is supposed to have 14 entries in it's color table (GDAL creates 19 based on "BinValues?" mapping).

[warmerdam]

I have found that the AIGRID driver does not fall through to PAM methods for the color table and color interpretation when native info is not available. Corrected in trunk (r22085). However, this does not seem to really be what Robin is requesting.

First let me say, that the binning logic seems to be doing the "right thing". That is the current logic returns an 19 color color-table for the value range 0-18, the real value range of the raster.

That said, if you need the code to do the wrong thing for some particular compatibility situation I'm willing to do what I can to help. Can you describe a bit more when you want padfBins to be ignored, so perhaps I can figure out an appropriate mechanism to suppress it's use? Should be always be the case for .aux files? Only for .aux files use for Arc/Info? binary grids? For all HFA (.img/.aux) files?

[gaopeng]

No, we don't want GDAL to do wrong thing. ArcGIS expects GDAL to remap the colormap using the lookup table. It assumes that the index # of the colortable is the pixel value.

Looking at it again, it turns out the problem in this case is that the correct colortable is returned from HFARasterBand, but there is another colortable in aux.xml (PAM) with only 14 entries, which overwrites the one from HFA.

Looking at the PAM/aux.xml file, it contains both a color table and an attribute table. I am not clear what's the definition of color table in PAM. Should it be always remapped as when saved or should it be remapped when it's read using the attribute table? Also the attribute table seems to be missing the pixel value.

[warmerdam]

The dataset I received did not have an .aux.xml file. I just forced generation of an .aux.xml file (gdal/swig/python/samples/histrep.py -req 0 100 101 -force g8_ui_nodata) and it produced a .aux.xml file with the correct colormap, histogram and RAT. The RAT included a column named BinValues? with a Usage of 5 which is GFU_MinMax.

The .aux.xml does end up with multiple data items (histogram, color table and RAT) all derived from the RAT in the .aux file which is kind of confusing but this is currently the expected behavior and shouldn't cause problems in most situations.

I am wondering if you might have had an old .aux.xml with the wrong color table generated before we fixed the HFA (.img/.aux) driver to support the BFUnique bin values properly?

(note I did my test with trunk including the r22085 fix, so behavior might be a bit different with 1.8-esri branch).

[gaopeng]

I attached another zip which contains all files associated with this grid. It's very likely the aux.xml was created before the fix.

A related note. We have found that the current color table model doesn't work well for colormaps with gaps, and negative values. For example, it's hard to represent the following colormap using GDAL color table

index   value   red    green  blue
0       0       0       0     0
1       100     255     0     100
2       400     0       255   0
....

The colortable for above colormap is large, and contains many filler colors. The problem will get worse if values go negative. Any idea for solving this? Currently ArcGIS writes a .clr file if it detects this type of colormaps.

[warmerdam]

On review of the .aux.xml in the new .zip it looks like it was generated before the BFUnique support was added. I'm guessing this means there is no further specific action item for this ticket?

For color tables, I quite agree the current model does not handle more generic needs well. One approach would be to just use RATs directly for more complex sorts of color tables and to disable the automatic RAT to GDALColorTable conversion in the HFA driver for cases it does not appear to handle well. Does that seem reasonable?

[rprinceley]

Frank,

There is a problem with CreateCopy? - ArcGIS display uses the RAT (14 entries) while CreateCopy? uses the color table (19 entries). Is it possible to match color table to RAT in this case?

[warmerdam]

I checked, and CreateCopy? should normally copy the RAT. However, the AIG driver was masking the RAT from the .img file since there was an AIG specific RAT implementation (from the Info directory). I have altered the code to fall through to PAM if there is no native RAT in trunk (r22103).

With this change, CreateCopy?() to a format like GeoTIFF will carry the RAT along too, and ArcGIS can choose to use the RAT in preference to the color table.

I have migrated both above changes back to 1.8-esri along with a few other improvements of Evens (r22104).

I think this addresses the AIG specific issues with exposing and copying RATs.