Opened 16 years ago
Closed 15 years ago
#2270 closed enhancement (fixed)
Patch for HDF5 block size recognition
Reported by: | jurien | Owned by: | chaitanya |
---|---|---|---|
Priority: | normal | Milestone: | 1.6.3 |
Component: | GDAL_Raster | Version: | |
Severity: | normal | Keywords: | HDF5 |
Cc: | warmerdam, dnadeau, antonio |
Description
Problem: HDF5 driver does not implement HDF chunk size detection. Simptoms: HDF5 is *very* slow on compressed files with chunksize != 1 line (default block size for HDF5 gdal driver)
I have modified the GDAL HDF5 driver to extract the chunk size and load it as the block size for the dataset.
Regards
Attachments (2)
Change History (15)
by , 16 years ago
Attachment: | gdal-1.5.0-HDF5-chunksize.patch added |
---|
comment:1 by , 16 years ago
Cc: | added |
---|---|
Component: | default → GDAL_Raster |
Keywords: | HDF5 added |
Milestone: | → 1.5.2 |
Mateusz,
As time permits please review and apply the patches. I am not aware autotests for HDF5. This should also be addressed as part of this effort.
comment:2 by , 16 years ago
Owner: | changed from | to
---|
comment:3 by , 16 years ago
To me it looks like the patch would lead to incorrect memory layout for border blocks.
comment:4 by , 16 years ago
@nowakpl: you mean if the blocksize is not a multiple of the imagesize? it is checked with at lines 304 and 308. I check for the size of the image and the boundaries of the block and keep the minimum of those. If that's not what you mean, please explain me what you meant so I can try and fix it.
Regards.
comment:5 by , 16 years ago
Owner: | removed |
---|
comment:6 by , 16 years ago
Hello,
1.5.3 is about to be released, has this patch been postponed?
Regards
comment:7 by , 16 years ago
Owner: | set to |
---|
Jurien,
I'm afraid this issue has fallen through the cracks. I'm not inclined to hold 1.5.3 back for this at this stage.
comment:8 by , 16 years ago
Cc: | added |
---|
I'm massively using a modified version of the HDF5 driver (build as plugin) that includes the patch from jurien.
If this helps I can confirm that the patch works very well and also produces a good performances boost. Thanks jurien.
follow-up: 11 comment:9 by , 15 years ago
Hello,
I'm bumping this ticket. I would really appreciate if the patch gets on the trunk.
To antonio: would you care to share your modifications on the HDF5 patch?
Regards.
comment:10 by , 15 years ago
Milestone: | 1.5.4 → 1.6.3 |
---|---|
Owner: | changed from | to
Jurien,
Could you provide a pointer to a file demonstrating the issue?
Chaitanya,
Could you review the patch, and apply if it works ok? Please try to add/extend autotest if there is a file utilizing this feature that is a reasonable size.
comment:11 by , 15 years ago
Replying to jurien:
Hello,
I'm bumping this ticket. I would really appreciate if the patch gets on the trunk.
To antonio: would you care to share your modifications on the HDF5 patch?
Regards.
Hi jurien, I've just uploaded an updated version of your patch. It should fix the issue at image borders and it should also work a little better with non-chunked datasets. It is just an extract of my working code so please check it before applying.
comment:12 by , 15 years ago
Status: | new → assigned |
---|
comment:13 by , 15 years ago
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
patch to read chunksize from an HDF5 file and optimize reads