[PATCH] HDF4, IReadBlock() and GDA_Update

[anssi]

I have an application which reads and writes an image using 2000 * nXSize chunk of data. The app works fine with GeoTIFF, but when using a HDF input, it produces an output with lines of missing data.

Please see the attached files for the source code. The two input and output sets are available here:

​https://www.transferbigfiles.com/13b17483-765b-4651-ad35-3c43526c3e0e?rid=ujdGk9lRuFMgI31WGP39GA%3d%3d

The link expires on Wed 27 Jul 7.20 AM CST

Thanks,

Anssi

[anssi]

source code

[anssi]

and gdal version 1.8.1 stable installed from source.

anssi

[Even Rouault]

Did you check first with gdal_translate to discriminate if it is a bug in your app or in the HDF driver (which driver by the way : HDF4 or HDF5 ?)

[anssi]

The original data was in hdf format. I extracted the band from the hdf file using gdal_translate and both, -of GeoTiff and -of HDF4Image. Both files look O.K. and the app seems to work O.K. with smaller chunk sizes that 2000.

-a

[Even Rouault]

OK, I've reproduced and diagnosed what happens. This is an issue with how the implementation of the HDF4Image driver interacts with the GDAL block cache mechanism. Depending on the size/fill rate of the block cache and the number of lines you read/write from your in/out datasets, when writing a HDF4Image, in some circumstances, the driver must refetch a partly written block from disk. But currently, the IReadBlock() method will not go to disk when it detects the dataset is in update mode (which is the case for a dataset in creation) and will then return a block initialized to 0. So that block will be rewritten but with 0's in its top lines. I've attached a fix that resolves your issue, but I'm not sure of its correctness in other use cases.

Perhaps Andrey will remember why this test was added and if we can safely remove it ?

There's a workaround however. If your chunk size is a multiple of the block height of the out dataset (124 in your case), there will be no opportunity to try re-reading an already written block, so that should avoid the problematic situation.

[anssi]

Thanks a lot,

I can confirm that the workaround works fine.

[Jukka Rahkonen]

Andrey never commented and because of that it is still not judged if the suggested patch to /trunk/gdal/frmts/hdf4/hdf4imagedataset.cpp would be good or not.

[Even Rouault]

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub. If you believe this ticket is still valid, you may file it to ​https://github.com/OSGeo/gdal/issues if it is not already reported there.