Opened 21 years ago

Last modified 20 years ago

#411 closed defect (fixed)

gdalinfo and unreferenced IMG files (hfa driver): all coordinates 0 instead of pixel values

Reported by: Markus Neteler Owned by: warmerdam
Priority: high Milestone:
Component: GDAL_Raster Version: unspecified
Severity: normal Keywords:
Cc: bradley@…

Description

Frank,

nothing severe, but probably not desired: I have received a XY IMG file
and gdalinfo reports all coordinates 0.

gdalinfo pirola_small_col.img
Driver: HFA/Erdas Imagine Images (.img)
Size is 2774, 2406
Coordinate System is:
LOCAL_CS[,
    UNIT[,1]]
Origin = (0.000000,0.000000)
Pixel Size = (0.000000,0.000000)
Corner Coordinates:
Upper Left  (   0.0000000,   0.0000000)
Lower Left  (   0.0000000,   0.0000000)
Upper Right (   0.0000000,   0.0000000)
Lower Right (   0.0000000,   0.0000000)
Center      (   0.0000000,   0.0000000)
Band 1 Block=64x64 Type=UInt16, ColorInterp=Undefined
  Overviews: 692x600, 345x299, 171x148, 84x73, 41x35
Band 2 Block=64x64 Type=UInt16, ColorInterp=Undefined
  Overviews: 692x600, 345x299, 171x148, 84x73, 41x35
Band 3 Block=64x64 Type=UInt16, ColorInterp=Undefined
  Overviews: 692x600, 345x299, 171x148, 84x73, 41x35
Band 4 Block=64x64 Type=UInt16, ColorInterp=Undefined
  Overviews: 692x600, 345x299, 171x148, 84x73, 41x35

gdal_translate -of gtiff pirola_small_pan.img pirola_small_pan.tif
Input file size is 2774, 2406
0...10...20...30...40...50...60...70...80...90...100 - done.

The resulting GeoTIFF file is OK.

Maybe below info is useful for you.


hfatest -dd pirola_small_pan.img
{1:lversion,1:LfreeList,1:LrootEntryPtr,1:sentryHeaderLength,1:LdictionaryPtr,}Ehfa_File,{1:Lnext,1:Lprev,1:Lparent,1:Lchild,1:Ldata,1:ldataSize,64:cname,32:ctype,1:tmodTime,}Ehfa_Entry,{16:clabel,1:LheaderPtr,}Ehfa_HeaderTag,{1:LfreeList,1:lfreeSize,}Ehfa_FreeListNode,{1:lsize,1:Lptr,}Ehfa_Data,{1:lwidth,1:lheight,1:e3:thematic,athematic,fft
of real-valued
data,layerType,1:e13:u1,u2,u4,u8,s8,u16,s16,u32,s32,f32,f64,c64,c128,pixelType,1:lblockWidth,1:lblockHeight,}Eimg_Layer,{1:lwidth,1:lheight,1:e3:thematic,athematic,fft
of real-valued
data,layerType,1:e13:u1,u2,u4,u8,s8,u16,s16,u32,s32,f32,f64,c64,c128,pixelType,1:lblockWidth,1:lblockHeight,}Eimg_Layer_SubSample,{1:e2:raster,vector,type,1:LdictionaryPtr,}Ehfa_Layer,{1:sfileCode,1:Loffset,1:lsize,1:e2:false,true,logvalid,1:e2:no
compression,ESRI GRID
compression,compressionType,}Edms_VirtualBlockInfo,{1:lmin,1:lmax,}Edms_FreeIDList,{1:lnumvirtualblocks,1:lnumobjectsperblock,1:lnextobjectnum,1:e2:no
compression,RLC
compression,compressionType,0:poEdms_VirtualBlockInfo,blockinfo,0:poEdms_FreeIDList,freelist,1:tmodTime,}Edms_State,{0:pcstring,}Emif_String,{1:oEmif_String,algorithm,0:poEmif_String,nameList,}Eimg_RRDNamesList,{1:oEmif_String,projection,1:oEmif_String,units,}Eimg_MapInformation,{1:oEmif_String,dependent,}Eimg_DependentFile,{1:oEmif_String,ImageLayerName,}Eimg_DependentLayerName,{1:lnumrows,1:lnumcolumns,1:e13:EGDA_TYPE_U1,EGDA_TYPE_U2,EGDA_TYPE_U4,EGDA_TYPE_U8,EGDA_TYPE_S8,EGDA_TYPE_U16,EGDA_TYPE_S16,EGDA_TYPE_U32,EGDA_TYPE_S32,EGDA_TYPE_F32,EGDA_TYPE_F64,EGDA_TYPE_C64,EGDA_TYPE_C128,datatype,1:e4:EGDA_SCALAR_OBJECT,EGDA_TABLE_OBJECT,EGDA_MATRIX_OBJECT,EGDA_RASTER_OBJECT,objecttype,}Egda_BaseData,{1:*bvalueBD,}Eimg_NonInitializedValue,{1:oEmif_String,fileName,2:LlayerStackValidFlagsOffset,2:LlayerStackDataOffset,1:LlayerStackCount,1:LlayerStackIndex,}ImgExternalRaster,{1:*bvalidFlags,}ImgValidFlags,{1:llayerStackCount,1:lwidth,1:lheight,1:lblockWidth,1:lblockHeight,1:e13:u1,u2,u4,u8,s8,u16,s16,u32,s32,f32,f64,c64,c128,pixelType,}ImgExternalLayerStackHeader,{1:LspaceUsedForRasterData,}ImgFormatInfo831,{1:bcovariance,}Esta_Covariance,{1:lnumrows,}Edsc_Table,{1:lnumRows,1:LcolumnDataPtr,1:e4:integer,real,complex,string,dataType,1:lmaxNumChars,}Edsc_Column,{1:x{0:pcstring,}Emif_String,type,0:pcMIFDictionary,0:pCMIFObject,}Emif_MIFObject,{1:lstartBin,1:LbinCount,1:LorigBinCount,1:x{1:x{0:pcstring,}Emif_String,type,0:pcMIFDictionary,0:pCMIFObject,}Emif_MIFObject,binFunction,1:x{0:pcstring,}Emif_String,title,}Edsc_BinFunction840,{0:px{0:pcstring,}Emif_String,LayerNames,1:*bExcludedValues,1:x{0:pcstring,}Emif_String,AOIname,1:lSkipFactorX,1:lSkipFactorY,1:x{1:x{0:pcstring,}Emif_String,type,0:pcMIFDictionary,0:pCMIFObject,}Emif_MIFObject,BinFunction,}Eimg_StatisticsParameters840,{1:dminimum,1:dmaximum,1:dmean,1:dmedian,1:dmode,1:dstddev,}Esta_Statistics,{1:dx,1:dy,}Eprj_Coordinate,{1:dwidth,1:dheight,}Eprj_Size,{0:pcproName,1:*oEprj_Coordinate,upperLeftCenter,1:*oEprj_Coordinate,lowerRightCenter,1:*oEprj_Size,pixelSize,0:pcunits,}Eprj_MapInfo,{1:lnumBins,1:e4:direct,linear,logarithmic,explicit,binFunctionType,1:dminLimit,1:dmaxLimit,1:*bbinLimits,}Edsc_BinFunction,{0:poEmif_String,LayerNames,1:*bExcludedValues,1:oEmif_String,AOIname,1:lSkipFactorX,1:lSkipFactorY,1:*oEdsc_BinFunction,BinFunction,}Eimg_StatisticsParameters830,.

HFADictionary:
HFAType Ehfa_File/18 bytes
    ULONG                 version[1];
    LONG                  freeList[1];
    LONG                  rootEntryPtr[1];
    USHORT                entryHeaderLength[1];
    LONG                  dictionaryPtr[1];

HFAType Ehfa_Entry/124 bytes
    LONG                  next[1];
    LONG                  prev[1];
    LONG                  parent[1];
    LONG                  child[1];
    LONG                  data[1];
    ULONG                 dataSize[1];
    UCHAR                 name[64];
    UCHAR                 type[32];
    TIME                  modTime[1];

HFAType Ehfa_HeaderTag/20 bytes
    UCHAR                 label[16];
    LONG                  headerPtr[1];

HFAType Ehfa_FreeListNode/8 bytes
    LONG                  freeList[1];
    ULONG                 freeSize[1];

HFAType Ehfa_Data/8 bytes
    ULONG                 size[1];
    LONG                  ptr[1];

HFAType Eimg_Layer/20 bytes
    ULONG                 width[1];
    ULONG                 height[1];
    ENUM                  layerType[1];
        thematic=0
        athematic=1
        fft of real-valued data=2
    ENUM                  pixelType[1];
        u1=0
        u2=1
        u4=2
        u8=3
        s8=4
        u16=5
        s16=6
        u32=7
        s32=8
        f32=9
        f64=10
        c64=11
        c128=12
    ULONG                 blockWidth[1];
    ULONG                 blockHeight[1];

HFAType Eimg_Layer_SubSample/20 bytes
    ULONG                 width[1];
    ULONG                 height[1];
    ENUM                  layerType[1];
        thematic=0
        athematic=1
        fft of real-valued data=2
    ENUM                  pixelType[1];
        u1=0
        u2=1
        u4=2
        u8=3
        s8=4
        u16=5
        s16=6
        u32=7
        s32=8
        f32=9
        f64=10
        c64=11
        c128=12
    ULONG                 blockWidth[1];
    ULONG                 blockHeight[1];

HFAType Ehfa_Layer/6 bytes
    ENUM                  type[1];
        raster=0
        vector=1
    LONG                  dictionaryPtr[1];

HFAType Edms_VirtualBlockInfo/14 bytes
    USHORT                fileCode[1];
    LONG                  offset[1];
    ULONG                 size[1];
    ENUM                  logvalid[1];
        false=0
        true=1
    ENUM                  compressionType[1];
        no compression=0
        ESRI GRID compression=1

HFAType Edms_FreeIDList/8 bytes
    ULONG                 min[1];
    ULONG                 max[1];

HFAType Edms_State/-1 bytes
    ULONG                 numvirtualblocks[1];
    ULONG                 numobjectsperblock[1];
    ULONG                 nextobjectnum[1];
    ENUM                  compressionType[1];
        no compression=0
        RLC compression=1
    Edms_VirtualBlockInfo p blockinfo[0];
    Edms_FreeIDList     p freelist[0];
    TIME                  modTime[1];

HFAType Emif_String/-1 bytes
    UCHAR               p string[0];

HFAType Eimg_RRDNamesList/-1 bytes
    Emif_String           algorithm[1];
    Emif_String         p nameList[0];

HFAType Eimg_MapInformation/-1 bytes
    Emif_String           projection[1];
    Emif_String           units[1];

HFAType Eimg_DependentFile/-1 bytes
    Emif_String           dependent[1];

HFAType Eimg_DependentLayerName/-1 bytes
    Emif_String           ImageLayerName[1];

HFAType Egda_BaseData/12 bytes
    ULONG                 numrows[1];
    ULONG                 numcolumns[1];
    ENUM                  datatype[1];
        EGDA_TYPE_U1=0
        EGDA_TYPE_U2=1
        EGDA_TYPE_U4=2
        EGDA_TYPE_U8=3
        EGDA_TYPE_S8=4
        EGDA_TYPE_U16=5
        EGDA_TYPE_S16=6
        EGDA_TYPE_U32=7
        EGDA_TYPE_S32=8
        EGDA_TYPE_F32=9
        EGDA_TYPE_F64=10
        EGDA_TYPE_C64=11
        EGDA_TYPE_C128=12
    ENUM                  objecttype[1];
        EGDA_SCALAR_OBJECT=0
        EGDA_TABLE_OBJECT=1
        EGDA_MATRIX_OBJECT=2
        EGDA_RASTER_OBJECT=3

HFAType Eimg_NonInitializedValue/8 bytes
    BASEDATA            * valueBD[1];

HFAType ImgExternalRaster/-1 bytes
    Emif_String           fileName[1];
    LONG                  layerStackValidFlagsOffset[2];
    LONG                  layerStackDataOffset[2];
    LONG                  layerStackCount[1];
    LONG                  layerStackIndex[1];

HFAType ImgValidFlags/8 bytes
    BASEDATA            * validFlags[1];

HFAType ImgExternalLayerStackHeader/22 bytes
    ULONG                 layerStackCount[1];
    ULONG                 width[1];
    ULONG                 height[1];
    ULONG                 blockWidth[1];
    ULONG                 blockHeight[1];
    ENUM                  pixelType[1];
        u1=0
        u2=1
        u4=2
        u8=3
        s8=4
        u16=5
        s16=6
        u32=7
        s32=8
        f32=9
        f64=10
        c64=11
        c128=12

HFAType ImgFormatInfo831/4 bytes
    LONG                  spaceUsedForRasterData[1];

HFAType Esta_Covariance/8 bytes
    BASEDATA              covariance[1];

HFAType Edsc_Table/4 bytes
    ULONG                 numrows[1];

HFAType Edsc_Column/14 bytes
    ULONG                 numRows[1];
    LONG                  columnDataPtr[1];
    ENUM                  dataType[1];
        integer=0
        real=1
        complex=2
        string=3
    ULONG                 maxNumChars[1];

HFAType Emif_String/0 bytes
    InlineType            {0:pcstring[1];

hfatest -dr pirola_small_pan.img
Raster Size = 2774 x 2406
Band 1: 64x64 tiles, type = 5
  Overview: 692x600 (blocksize 64x64)
  Overview: 345x299 (blocksize 64x64)
  Overview: 171x148 (blocksize 64x64)
  Overview: 84x73 (blocksize 64x64)
  Overview: 41x35 (blocksize 41x35)
  Min: 0   Max: 0   Mean: 0
  Median: 0   Mode: 0   Stddev: 0
MapInfo.proName =
MapInfo.upperLeftCenter.x = 0.00
MapInfo.upperLeftCenter.y = 0.00

hfatest -dt pirola_small_pan.img
root(root) @ 82 + 0 @ 0

  IMGFormatInfo(ImgFormatInfo831) @ 2632 + 4 @ 2760
  + spaceUsedForRasterData = 13697024

  Layer_1(Eimg_Layer) @ 210 + 20 @ 338
  + width = 2774
  + height = 2406
  + layerType = athematic
  + pixelType = u16
  + blockWidth = 64
  + blockHeight = 64

    Map_Info(Eprj_MapInfo) @ 1732 + 102 @ 1860

    StatisticsParameters(Eimg_StatisticsParameters830) @ 1204 + 84 @ 1962

    Statistics(Esta_Statistics) @ 1028 + 48 @ 1156

    HistogramParameters(Eimg_StatisticsParameters830) @ 900 + 84 @ 2346

    Descriptor_Table(Edsc_Table) @ 494 + 4 @ 622
    + numrows = 34114

      #Bin_Function#(Edsc_BinFunction) @ 2188 + 30 @ 2316

      #Bin_Function840#(Edsc_BinFunction840) @ 630 + 273365 @ 13542860

      Histogram(Edsc_Column) @ 758 + 14 @ 886
      + numRows = 34114
      + columnDataPtr = 13818755
      + dataType = real
      + maxNumChars = 0

      Value(Edsc_Column) @ 2046 + 14 @ 2174
      + numRows = 34114
      + columnDataPtr = 14368763
      + dataType = integer
      + maxNumChars = 0

    Eimg_NonInitializedValue(Eimg_NonInitializedValue) @ 26342 + 22 @ 26470
    + valueBD = (basedata)

    RasterDMS(Edms_State) @ 2768 + 23446 @ 2896
    + numvirtualblocks = 1672
    + numobjectsperblock = 4096
    + nextobjectnum = 0
    + compressionType = RLC compression
    + blockinfo[0] =
    +     fileCode = 0
    +     offset = 26492
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[1] =
    +     fileCode = 0
    +     offset = 42876
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[2] =
    +     fileCode = 0
    +     offset = 59260
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[3] =
    +     fileCode = 0
    +     offset = 75644
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[4] =
    +     fileCode = 0
    +     offset = 92028
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[5] =
    +     fileCode = 0
    +     offset = 108412
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[6] =
    +     fileCode = 0
    +     offset = 124796
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[7] =
    +     fileCode = 0
    +     offset = 141180
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[8] =
    +     fileCode = 0
    +     offset = 157564
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[9] =
    +     fileCode = 0
    +     offset = 173948
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[10] =
    +     fileCode = 0
    +     offset = 190332
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[11] =
    +     fileCode = 0
    +     offset = 206716
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[12] =
    +     fileCode = 0
    +     offset = 223100
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[13] =
    +     fileCode = 0
    +     offset = 239484
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[14] =
    +     fileCode = 0
    +     offset = 255868
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    + blockinfo[15] =
    +     fileCode = 0
    +     offset = 272252
    +     size = 8192
    +     logvalid = true
    +     compressionType = no compression
    +  ... remaining instances omitted ...
    + freelist = (no values)
    + modTime = 1954545664

    Ehfa_Layer(Ehfa_Layer) @ 358 + 6 @ 486
    + type = raster
    + dictionaryPtr = 2608

    RRDNamesList(Eimg_RRDNamesList) @ 1332 + 272 @ 1460
    + algorithm =
    +     string = `IMAGINE 2X2 Resampling'
    + nameList[0] =
    +     string = `pirola_small_pan.rrd(:Layer_1:_ss_4_)'
    + nameList[1] =
    +     string = `pirola_small_pan.rrd(:Layer_1:_ss_8_)'
    + nameList[2] =
    +     string = `pirola_small_pan.rrd(:Layer_1:_ss_16_)'
    + nameList[3] =
    +     string = `pirola_small_pan.rrd(:Layer_1:_ss_32_)'
    + nameList[4] =
    +     string = `pirola_small_pan.rrd(:Layer_1:_ss_64_)'

Change History (8)

comment:1 by warmerdam, 21 years ago

Markus,

There is something pretty funky about the MapInfo node for this file.  It
may well indicate a low level parsing error in my HFA code.  Do you still have
this file?  Is it possible for me to get a copy of it? 


comment:2 by neteler@…, 21 years ago

Frank,

how should I upload? It's some MB... To the bugtracker?

Markus

comment:3 by warmerdam, 21 years ago

Markus, 

Not in the bug tracker.

You can ftp it to pub/incoming on gdal.velocet.ca and let me know then
name when it is done. 

Thanks,

comment:4 by neteler@…, 21 years ago

Frank,

the files are uploaded:
image_col.img
image_col.rrd

Best regards

 Markus

comment:5 by neteler@…, 20 years ago

Frank,

I received more ERDAS files with the described problem.
The customer states that his ERDAS version is capable to
print out the coordinates (instead of 0).
With GDAL I get with X,Y coordinates instead of the map
coordinates (however, the uploaded files are missing projection
info, may that cause the problem?). Like that maps with
different resolutions in that file don't match any more spatially.

Best regards

 Markus Neteler


comment:6 by warmerdam, 20 years ago

Markus, 

I looked into the image_col.img file (finally!) and found that the data
dictionary doesn't have an entry for Eprj_MapInfo.  I think I'll add some 
capability to load missing definitions from a file distributed with GDAL.  I 
have no idea why Imagine would be generating some files missing these definitions.
Quite annoying. 


comment:7 by warmerdam, 20 years ago

OK,

I figured out that sections like the following in the data dictionary are
confusing the hfadictionary parser. 

{1:x{0:pcstring,}Emif_String,type,0:pcMIFDictionary,0:pCMIFObject,
}Emif_MIFObject,{1:lstartBin,1:LbinCount,1:LorigBinCount,1:x{1:x{0:pcstring,}Emi
f_String,type,0:pcMIFDictionary,0:pCMIFObject,}Emif_MIFObject,binFunction,1:x{0:
pcstring,}Emif_String,title,}Edsc_BinFunction840,{0:px{0:pcstring,}Emif_String,L
ayerNames,1:*bExcludedValues,1:x{0:pcstring,}Emif_String

It looks like the dictionary is corrupt or else there is some new nesting
scheme in Image 8.4 that I don't understand. 

In any event I have modified HFADictionary::HFADictionary() to define a few
key types if they are missing from the data dictionary.  I have also modified
HFAType::Initialize() to try and recover if a dictionary entry appears
unparsable.  Between these two changes things seem much better now in CVS.

I am closing this now, but I may need to revisit the problem in the future
with a "proper" fix for dictionary reading if I find there is a pattern to it
all.


comment:8 by warmerdam, 20 years ago

*** Bug 476 has been marked as a duplicate of this bug. ***
Note: See TracTickets for help on using tickets.