GeoTiff doesn't read .RRD overviews

[gaopeng]

I have a tiff image with associated .aux and .rrd files. The tiff image doesn't have any overviews. The following is print out from gdalinfo:

C:\Program Files\FWTools2.2.8>gdalinfo \\esri.com\toybox\public\jcalkins\GeoRec
ified\po_20050503_pan.tif
Driver: GTiff/GeoTIFF
Files: \\esri.com\toybox\public\jcalkins\GeoRectified\po_20050503_pan.tif
       \\esri.com\toybox\public\jcalkins\GeoRectified\po_20050503_pan.aux
Size is 19361, 16156
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.2572235630016,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433],
    AUTHORITY["EPSG","4326"]]
Origin = (168.640342000000000,-44.948828499999998)
Pixel Size = (0.000008993356320,-0.000008993216060)
Metadata:
  AREA_OR_POINT=Area
  TIFFTAG_SOFTWARE=IMAGINE TIFF Support
Copyright 1991 - 1999 by ERDAS, Inc. All Rights Reserved
@(#)$RCSfile: etif.c $ $Revision: 1.10.1.9.1.9.2.11 $ $Date: 2004/09/15 18:42:0
EDT $
  TIFFTAG_XRESOLUTION=1
  TIFFTAG_YRESOLUTION=1
  TIFFTAG_RESOLUTIONUNIT=1 (unitless)
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  ( 168.6403420, -44.9488285) (168d38'25.23"E, 44d56'55.78"S)
Lower Left  ( 168.6403420, -45.0941229) (168d38'25.23"E, 45d 5'38.84"S)
Upper Right ( 168.8144624, -44.9488285) (168d48'52.06"E, 44d56'55.78"S)
Lower Right ( 168.8144624, -45.0941229) (168d48'52.06"E, 45d 5'38.84"S)
Center      ( 168.7274022, -45.0214757) (168d43'38.65"E, 45d 1'17.31"S)
Band 1 Block=64x64 Type=UInt16, ColorInterp=Gray
  Min=0.000 Max=2047.000
  Minimum=0.000, Maximum=2047.000, Mean=374.448, StdDev=294.031
  Metadata:
    STATISTICS_MINIMUM=0
    STATISTICS_MAXIMUM=2047
    STATISTICS_MEAN=374.44761883001
    STATISTICS_MEDIAN=3.4506643385542e-183
    STATISTICS_MODE=5.4814051525109e-294
    STATISTICS_STDDEV=294.03057907622
    LAYER_TYPE=athematic

C:\Program Files\FWTools2.2.8>

[warmerdam]

Gao,

I have attempted to reproduce this problem unsuccessfully. I created a tiff with an .aux and .rrd with gdaladdo and then did:

Driver: GTiff/GeoTIFF
Files: \\gdal\warmerda\wrk.tif
       \\gdal\warmerda\wrk.aux
       \\gdal\warmerda\wrk.rrd
Size is 512, 512
Coordinate System is:
PROJCS["NAD27 / UTM zone 11N",
    GEOGCS["NAD27",
        DATUM["North_American_Datum_1927",
            SPHEROID["Clarke 1866",6378206.4,294.9786982139006,
                AUTHORITY["EPSG","7008"]],
            AUTHORITY["EPSG","6267"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4267"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",-117],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AUTHORITY["EPSG","26711"]]
Origin = (440720.000000000000000,3751320.000000000000000)
Pixel Size = (60.000000000000000,-60.000000000000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  440720.000, 3751320.000) (117d38'28.21"W, 33d54'8.47"N)
Lower Left  (  440720.000, 3720600.000) (117d38'20.79"W, 33d37'31.04"N)
Upper Right (  471440.000, 3751320.000) (117d18'32.07"W, 33d54'13.08"N)
Lower Right (  471440.000, 3720600.000) (117d18'28.50"W, 33d37'35.61"N)
Center      (  456080.000, 3735960.000) (117d28'27.39"W, 33d45'52.46"N)
Band 1 Block=512x16 Type=Byte, ColorInterp=Gray
  Overviews: 256x256
  Metadata:
    LAYER_TYPE=athematic

Note that the .rrd file is reference, and the band has an overview.

I put the test data I used at: ​http://home.gdal.org/tmp/bug3463_works.zip

Perhaps you could confirm that the overviews come through in this case?

I think I will need the .tif/.aux/.rrd that is not working to dig further. I suspect something is unusual about the .aux or .rrd file.

[gaopeng]

Frank,

Yes. I can see the overview with your test dataset. I think the overviews in my case are in the .rrd file. I can see the following is called twice: .tif -> .aux -> 0. It didn't find the .rrd file.

I uploaded a test dataset: ​ftp://GDAL@ftp.esri.com/Ticket3463.zip

/* -------------------------------------------------------------------- */
/*      We didn't find that, so try and find a corresponding aux        */
/*      file.  Check that we are the dependent file of the aux          */
/*      file.                                                           */
/* -------------------------------------------------------------------- */
    if( !poODS && !EQUAL(pszBasename,":::VIRTUAL:::") )
    {
        poODS = GDALFindAssociatedAuxFile( pszBasename, poDS->GetAccess(),
                                           poDS );

[warmerdam]

Ok, problem confirmed.

The .aux file in this case has no RRDNamesList. I'm not sure how valid this is, but it seems you want it to work so I have modified the overview search code to look for an .rrd file even if it is not referenced via RRDNamesList. It hooks up with existing logic for using unreferenced overviews within the .img or .aux file.

The changes are in trunk (r19041) and 1.6-esri (r19042).

Let me know if the problem persists.

[gaopeng]

I made change from RRD full path name, to, just RRD file name, to make it work.

        CPLString osFullRRD = CPLFormFilename( psInfo->pszPath, osRRDFilename,
                                               NULL );

I have lots of tiffs with .RRD files like this. I am not sure either how valid these are. Once opened, I can see 7 levels of overviews. But the first level overview starts at 256x.

[warmerdam]

Ah, I see the problem with the path handling. I really need to do one test each time *not* in the same directory as the target file.

I have fixed this in trunk (r19043) and 1.6-esri branch (r19044).

The overviews are recorded in the order they are encountered. Does this cause you problems? I can sort them by sizing if you need though in general GDAL does not guaranteed that GetOverview() returns them in any particular order.

[gaopeng]

It will be great if you could order the overviews on size. ArcGIS currently assumes they are ordered.

[warmerdam]

OK, the autodiscover code now orders the overviews in trunk (r19045,19046) and 1.6-esri (r19047).

Note this still does not guarantee good ordering of overviews from all parts of GDAL. If ordering is important to the application it would be prudent eventually to order them at the application level.

[gaopeng]

The attached test .lan image causes a crash in the new RRD autodiscover code.

[warmerdam]

The problem was that the .rrd file lacked entries for some of the base bands and the code was careless about accessing through the resulting null pointer. Issue corrected in trunk (r19222) and 1.6-esri(r19223).