Subdataset metadata unavalable

[michaeldg]

I have mostly been using gdal through the python interface. However i believe the problem is with gdal or one of its dependent libraries.

Reproducing the bug:

Download a MODIS 1A granual from nasa. At the time of this bug report this was available:

wget ​ftp://ladsweb.nascom.nasa.gov/allData/5/MOD01/2008/096/MOD01.A2008096.0020.005.2008158071242.hdf

python

import gdal

/usr/lib/python2.5/site-packages/GDAL-1.5.0-py2.5-linux-x86_64.egg/osgeo/gdal.py:81: DeprecationWarning: gdal.py was placed in a namespace, it is now available as osgeo.gdal

DeprecationWarning)

ds = gdal.Open('MOD01.A2008096.0020.005.2008158071242.hdf') print ds.GetSubDatasets()

[]

print ds.GetMetadata('SubDatasets')

{}

Expected Result:

List of subdatasets (on previous versions)

This bug is also present when using gdal 1.4.4 on a fresh install on Ubuntu Hardy Heron 8.4 64bit where we are using it.

Even when using command line gdalinfo there is no listing of the subdatasets:

Driver: HDF4/Hierarchical Data Format Release 4 Files: MOD01.A2008096.0020.005.2008158071242.hdf Size is 512, 512 Coordinate System is `' Metadata:

Number of Scans=-889192448 Number of Day mode scans=-889192448 Number of Night mode scans=0 Max Total Frames=-368771072 Max Earth Frames=1241841664 Max SD Frames=838860800 Max SRCA Frames=167772160 Max BB Frames=838860800 Max SV Frames=838860800 Scan Types in product=Day Incomplete Scans=0 Missing Packets=0 Packets with bad CRC=0 Discarded packets=0 LOCALGRANULEID=MOD01.A2008096.0020.005.2008158071242.hdf PRODUCTIONDATETIME=2008-06-06T07:16:16.000Z DAYNIGHTFLAG=Day REPROCESSINGACTUAL=reprocessed once LOCALVERSIONID=5.0.0 REPROCESSINGPLANNED=further update is anticipated EQUATORCROSSINGDATE=2008-04-05 EQUATORCROSSINGTIME=00:47:27.183320 ORBITNUMBER=44134 EQUATORCROSSINGLONGITUDE=145.655356216241 VERSIONID=5 SHORTNAME=MOD01 INPUTPOINTER=ENG_DATA_LIST_TERRA_V5.0.30.6, MOD00F.A2008096.0015.20080960445.001.PDS GRINGPOINTLONGITUDE=-10.8560461673946, -89.1750911385304, -141.865539505048, 134.572869066794 GRINGPOINTLATITUDE=79.6701346647909, 68.794482876379, 69.2706455207659, 81.2141153131848 GRINGPOINTSEQUENCENO=1, 2, 3, 4 EXCLUSIONGRINGFLAG=N RANGEENDINGDATE=2008-04-05 RANGEENDINGTIME=00:25:00.000000 RANGEBEGINNINGDATE=2008-04-05 RANGEBEGINNINGTIME=00:20:00.000000 PGEVERSION=5.0.30 ASSOCIATEDSENSORSHORTNAME=MODIS ASSOCIATEDPLATFORMSHORTNAME=Terra ASSOCIATEDINSTRUMENTSHORTNAME=MODIS GRANULENUMBER=6 PROCESSVERSION=5.0.5 EASTBOUNDINGCOORDINATE=180.0 NORTHBOUNDINGCOORDINATE=89.991583397565 SOUTHBOUNDINGCOORDINATE=68.7951988748994 WESTBOUNDINGCOORDINATE=-180.0 DESCRREVISION=5.0 PRODUCTIONHISTORY=PGE01:5.0.30 LONGNAME=MODIS/Terra Raw Radiances in Counts 5-Min L1A Swath PROCESSINGENVIRONMENT=Linux minion5256 2.6.20.3 #1 SMP Thu Mar 22 09:36:24 EST 2007 i686 IntelR XeonR CPU 5148 @ 2.33GHz unknown GNU/Linux

Corner Coordinates: Upper Left ( 0.0, 0.0) Lower Left ( 0.0, 512.0) Upper Right ( 512.0, 0.0) Lower Right ( 512.0, 512.0) Center ( 256.0, 256.0)

[warmerdam]

Michael,

I think I will need access to a dataset demonstrating this problem. Can you attach a small file, or provide an http pointer to a file demonstrating the issue?

[warmerdam]

Umm, forget the last request, I see a pointer to the dataset is provided. I'll try and look into this in the next few days.

[warmerdam]

Unfortunately, today the referenced granule is no longer available. Could you point me to some new sample data demonstrating the issue?

[michaeldg]

Nasa is constantly yanking data off thier site. I think any level 1A (digital counts) will have the same problem. Every granule we have looked at in

​ftp://ladsweb.nascom.nasa.gov/allData/5/MOD01/

has exactly the same problem. I haven't looked at other level data to be honest or tested for Modis on Aqua but I wouldn't be surprised if it failed there too. I posted as major because any code (and we have some) that depends on pulling the names out of the DataSubsets is now broken.

This will probably stay up for a view days:

​ftp://ladsweb.nascom.nasa.gov/allData/5/MOD01/2008/096/MOD01.A2008096.0205.005.2008157022341.hdf

[michaeldg]

Any success at reproducing the error?

[warmerdam]

I downloaded MOD01.A2008037.0840.005.2008165030430.hdf and when I run gdalinfo on it I get no subdatasets. With debug enabled I also see the message:

HDF4: Number of HDF-EOS swaths: 0
HDF4: Number of HDF-EOS grids: 0

This leads me to wonder if this dataset is representative of your problem, and or whether we have broken support for HDF files without swaths or grids (ie. files with just SDSes). At this point I got somewhat confused and distracted by something else and didn't get back to it. Could you run gdalinfo with the "--debug on" switch and confirm if you see the message about there being zero swaths and grids?

Also, I tried with my local build from the gdal 1.5 and 1.4 branches and they both seemed to report the same thing. I don't know if that is because the branches incorporate some bug fix that broke things or not.

[michaeldg]

I can confirm that I do not see the subdatasets through gdalinfo. I suspect this problem has nothing to do with the python binding but as you say, the interface between gdal and hdf4 libraries. We have a preupgrade version, gdal 1.3.2 that shows the data subsets. I definitely have some granules that show the number of grids and swaths correctly but still don't show the data subsets.

[Kyle Shannon]

gdalinfo now reports subdatasets from this file:

​ftp://ladsweb.nascom.nasa.gov/allData/5/MOD01/2008/001/MOD01.A2008001.0950.005.2014303181955.hdf

closing as fixed at some point.