Opened 13 years ago

Closed 2 years ago

#2758 closed defect (wontfix)

HDF4 reading MISR Level 3 with incorrect dimension

Reported by: ilucena Owned by: warmerdam
Priority: normal Milestone: closed_because_of_github_migration
Component: GDAL_Raster Version: unspecified
Severity: normal Keywords: HDF4
Cc: rprinceley

Description

The first sub-dataset indicates the existence of 4 dimensions as shown by gdalinfo:

% gdalinfo MISR_AM1_CGAS_MAY_2007_F15_0031.hdf
...
Subdatasets:
  SUBDATASET_1_NAME=HDF4_EOS:EOS_GRID:"MISR_AM1_CGAS_MAY_2007_F15_0031.hdf":AerosolParameterAverage:Absorbing optical depth
  SUBDATASET_1_DESC=[360x720x9x4] Absorbing optical depth AerosolParameterAverage (32-bit floating-point)
...

As I translate the whole sub-dataset to GTIFF it creates 9 bands:

% gdal_translate "HDF4_EOS:EOS_GRID:MISR_AM1_CGAS_MAY_2007_F15_0031.hdf:AerosolParameterAverage:Absorbing optical depth" May_2007.tif -sds
Input file size is 720, 360
0...10...20...30...40...50...60...70...80...90...100 - done.

% gdalinfo May_2007.tif
...
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (-180.0000000,  90.0000000) (180d 0'0.00"W, 90d 0'0.00"N)
Lower Left  (-180.0000000, -90.0000000) (180d 0'0.00"W, 90d 0'0.00"S)
Upper Right ( 180.0000000,  90.0000000) (180d 0'0.00"E, 90d 0'0.00"N)
Lower Right ( 180.0000000, -90.0000000) (180d 0'0.00"E, 90d 0'0.00"S)
Center      (   0.0000000,   0.0000000) (  0d 0'0.01"E,  0d 0'0.01"N)
Band 1 Block=720x1 Type=Float32, ColorInterp=Gray
  NoData Value=-9999
Band 2 Block=720x1 Type=Float32, ColorInterp=Undefined
  NoData Value=-9999
Band 3 Block=720x1 Type=Float32, ColorInterp=Undefined
  NoData Value=-9999
Band 4 Block=720x1 Type=Float32, ColorInterp=Undefined
  NoData Value=-9999
Band 5 Block=720x1 Type=Float32, ColorInterp=Undefined
  NoData Value=-9999
Band 6 Block=720x1 Type=Float32, ColorInterp=Undefined
  NoData Value=-9999
Band 7 Block=720x1 Type=Float32, ColorInterp=Undefined
  NoData Value=-9999
Band 8 Block=720x1 Type=Float32, ColorInterp=Undefined
  NoData Value=-9999
Band 9 Block=720x1 Type=Float32, ColorInterp=Undefined
  NoData Value=-9999

But by looking at the image it seams like it got dimensions wrong. There are just vertical stripes along the image.

Sample dataset are available at:

http://eosweb.larc.nasa.gov/PRODOCS/misr/level3/level3_CGAS_small.html

This particular one came from:

ftp://l4ftl01.larc.nasa.gov/MISR/MIL3MAE.004/2007.05.01/

Attachments (1)

OpenEVScreenshot.PNG (41.8 KB) - added by ilucena 13 years ago.
OpenEV screenshot of GTIFF file

Download all attachments as: .zip

Change History (6)

Changed 13 years ago by ilucena

Attachment: OpenEVScreenshot.PNG added

OpenEV screenshot of GTIFF file

comment:1 Changed 7 years ago by rprinceley

Cc: rprinceley added

comment:2 Changed 7 years ago by Jukka Rahkonen

I do not understand HDF4 at all but I have a feeling that some misunderstanding was also a reason for this ticket.

The dimensions of the selected subdataset are, as reported [360x720x9x4] GDAL decides that the two first dimensions are the X and Y sizes of the raster and the third dimension is telling the number of bands.

This theory can be verified by converting another subset "AerosolParameterAverage:Algorithm type count" [360x720x10x2] into GeoTIFF. It contains 10 bands.

I believe that the meaning of -sds parameter of gdal_translate is understood in a wrong way. It does not stand for "the whole subdataset" but "all the subdatasets". This is from gdal_tranlate manual page:

Copy all subdatasets of this file to individual output files. 
Use with formats like HDF or OGDI that have subdatasets. 
The output file naming scheme has changed in GDAL 1.11 
(e.g. ofile_1.tif, ofile_2.tif).

In the example command -sds has no effect because one subdataset is explicitly selected. I do not know how -sds should be used. I thought that this command would create 9 separate tiff files but it did not work for me:

gdal_translate "HDF4_EOS:EOS_GRID:MISR_AM1_CGAS_MAY_2007_F15_0031.hdf" -sds ofile.tif
GDALOpen failed - 0

I consider that there is not a bug as the title is expressing it. There may be some fuzzy logic in detecting the number of bands and missing features with 4 and more dimensions. It seems to me that user has no control on which dimension is used for splitting data into bands. That's probably worth a new ticket. Perhaps this NetCDF patch is having a similar aim #2540.

I keep this ticket open for some weeks and wait that ilucena or someone else who understands HDF4 better that I makes a new review. If there will be no activity I am planning to close this ticket as invalid.

comment:3 Changed 7 years ago by ilucena

You may be right to say that the -sds parameter has no effect on my example. I don't think the -sds will work if the output format doesn't support subdatasets and it will not going to create several .tif files, as far as I know.

The HDF format is very flexible and in some cases the GDAL driver is able to identify the data provider/product to make the correct decision about the dimensions.

But as far as I know there is no way for a user to inform the HDF4 driver how they want the dimensions to be processed http://www.gdal.org/frmt_hdf4.html.

So maybe the solution is to follow the NetCDF example.

comment:4 Changed 7 years ago by Jukka Rahkonen

Option -sds should create many tiff files. See http://trac.osgeo.org/gdal/ticket/5119. Perhaps is is somehow broken or all subdatasets should have similar dimensions etc. Do you have HDF files to test with?

comment:5 Changed 2 years ago by Even Rouault

Milestone: closed_because_of_github_migration
Resolution: wontfix
Status: newclosed

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub?. If you believe this ticket is still valid, you may file it to https://github.com/OSGeo/gdal/issues if it is not already reported there.

Note: See TracTickets for help on using tickets.