Opened 13 years ago

Closed 13 years ago

#3879 closed enhancement (fixed)

Performance improvment for gdal_retile

Reported by: mcr Owned by: Even Rouault
Priority: normal Milestone: 1.8.0
Component: Utilities Version: unspecified
Severity: normal Keywords: performance gdal_retile directory structure
Cc:

Description

For large images, some file systems have performance problems if the number of files in a directory is to big, causing gdal_retile not to finish in reasonable time.

A new -useDirForEachRow parameter creates a different output structure. The tiles of the base image are stored in a subdirectory called 0, the pyramids in subdirectories numbered 1,2,.... Within each of these directories another level of subdirectories is created, numbered from 0...n, depending of how many tile rows are needed for each level. Finally, a directory contains only the the tiles for one row for a specific level. For large images a performance improvement of a factor N could be achieved.

A user reported a gdal_retile job not finishing within 2 weeks. After this patch, the job needs 2 days.

Attachments (1)

gdal_retile.patch (9.3 KB ) - added by mcr 13 years ago.

Download all attachments as: .zip

Change History (3)

by mcr, 13 years ago

Attachment: gdal_retile.patch added

comment:1 by warmerdam, 13 years ago

Owner: changed from warmerdam to Even Rouault

Even,

I assume you will want to apply this since you have been doing work on gdal_retile.py? Bounce it back if that is not the case.

comment:2 by Even Rouault, 13 years ago

Resolution: fixed
Status: newclosed

Patch applied in r21279. I did not apply the changes in the autotest that are unnecessary. I'm not sure why you needed them, perhaps some floating point difference, and they are anyway not related to the content of the patch

Note: See TracTickets for help on using tickets.