Opened 12 years ago
Closed 12 years ago
#3879 closed enhancement (fixed)
Performance improvment for gdal_retile
Reported by: | mcr | Owned by: | Even Rouault |
---|---|---|---|
Priority: | normal | Milestone: | 1.8.0 |
Component: | Utilities | Version: | unspecified |
Severity: | normal | Keywords: | performance gdal_retile directory structure |
Cc: |
Description
For large images, some file systems have performance problems if the number of files in a directory is to big, causing gdal_retile not to finish in reasonable time.
A new -useDirForEachRow parameter creates a different output structure. The tiles of the base image are stored in a subdirectory called 0, the pyramids in subdirectories numbered 1,2,.... Within each of these directories another level of subdirectories is created, numbered from 0...n, depending of how many tile rows are needed for each level. Finally, a directory contains only the the tiles for one row for a specific level. For large images a performance improvement of a factor N could be achieved.
A user reported a gdal_retile job not finishing within 2 weeks. After this patch, the job needs 2 days.
Attachments (1)
Change History (3)
by , 12 years ago
Attachment: | gdal_retile.patch added |
---|
comment:1 by , 12 years ago
Owner: | changed from | to
---|
comment:2 by , 12 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
Patch applied in r21279. I did not apply the changes in the autotest that are unnecessary. I'm not sure why you needed them, perhaps some floating point difference, and they are anyway not related to the content of the patch
Even,
I assume you will want to apply this since you have been doing work on gdal_retile.py? Bounce it back if that is not the case.