Opened 15 years ago

Closed 13 years ago

#2208 closed defect (fixed)

Slow access to compressed HDF4

Reported by: nowakpl Owned by: warmerdam
Priority: normal Milestone: 1.8.0
Component: GDAL_Raster Version: svn-trunk
Severity: normal Keywords: HDF4


The problem is GDAL accessing the data one line at a time. I made a quick and dirty fix by setting nBlockYSize to 128 and some additional code. This is by no means a proper fix. time /opt/gdal-r13204_x86-64_ubuntu7/bin/gdal_translate HDF4_EOS:EOS_SWATH:"MOD02QKM.A2008038.0210.005.2008038092447.hdf":MODIS_SWATH_Type_L1B:EV_250_RefSB test.tiff Input file size is 5416, 8120 0...10...20...30...40...50...60...70...80...90...100 - done. real 2m53.651s user 2m50.450s

Attachments (1)

hdf4.patch (6.6 KB ) - added by nowakpl 15 years ago.

Download all attachments as: .zip

Change History (5)

by nowakpl, 15 years ago

Attachment: hdf4.patch added

comment:1 by warmerdam, 15 years ago

Component: defaultGDAL_Raster
Keywords: HDF4 added
Milestone: 1.6.0
Resolution: fixed
Status: newclosed

I have encorporated a variation on this patch in trunk (r13785) but only operating on SDS datasets (all I can easily test just now). Given testing it could be easily applied to other cases, and likely back ported to 1.5 branch.


comment:2 by jluis, 15 years ago

Version: unspecifiedsvn-trunk

I'm reopening this because I encountered a very slow performance using the example file below. The comparisons are done against Matlab and a converted to geotiff version of the same file.

Joaquim Luis

The example file can be get from

The timings below (in seconds) respect my MEX for doing gdal read inside Matlab. That program is called gdalread. Hardware is a 2.5 GHz core 2 duo

Let tt.hdf = 199101.s04m1pfv50-sst-16b.hdf

tic,Z = gdalread('c:\tt.hdf');toc elapsed_time = 21.5

However, using the internal Matlab solution one get:

tic,sst=hdfread('c:\tt.hdf', 'sst', 'index', {[1 1],[1 1], [4096 8192]});toc elapsed_time = 0.8120

Just to show that the problem is not in "gdalread" I converted the hdf file to getiff (lixo.tiff), which took about 25 sec

tic,Z = gdalread('c:\lixo.tiff');toc elapsed_time = 2.0930

The slow performance is noticeable when fetching only the file's metadata

tic,Z = gdalread('c:\tt.hdf','-M');toc elapsed_time = 1.0310

% Matlab's way tic,Z=hdfinfo('c:\tt.hdf');toc elapsed_time = 0.0310

% On the geotiff file tic,Z = gdalread('c:\lixo.tiff','-M');toc elapsed_time = 0.0310

comment:3 by Even Rouault, 15 years ago

Resolution: fixed
Status: closedreopened

comment:4 by Even Rouault, 13 years ago

Resolution: fixed
Status: reopenedclosed

r18964 /trunk/gdal/frmts/hdf4/hdf4imagedataset.cpp: HDF4 : speed up access to HDF4_SDS datasets; allow multi-line block dimension for HDF4_EOS datasets (#2208)

r18965 /trunk/autotest/gcore/ Add testing of HDF4 datasets available on

r18966 /trunk/autotest/gcore/ HDF4 : Test faster access to 199101.s04m1pfv50-sst-16b.hdf (#2208)

Note: See TracTickets for help on using tickets.