Opened 11 years ago
Last modified 5 years ago
#1938 reopened enhancement
r.fillnulls: per hole filling speed-up
| Reported by: | sbl | Owned by: | |
|---|---|---|---|
| Priority: | normal | Milestone: | 7.6.2 |
| Component: | Python | Version: | svn-trunk |
| Keywords: | r.fillnulls | Cc: | |
| CPU: | All | Platform: | All |
Description
When running on a large grid (60,000 x 50,000 cells) with lots of NoData areas (52,000 holes) r.fillnulls is very slow. I figured out, that there was significant potential for speedup in the way r.fillnulls applies region cropping (the way it moves the region from hole to hole).
Therefore I would like to propose a speedup provided by the attached patch.
Main difference is, that the raster based region cropping (which ran a map calculator expression on (nearly) the entire input raster) within the loop over the holes is replaced by a vector based approach (similar to the one used in r.connectivity.distance). Here, before r.fillnulls loops over the holes, the latter are converted to vector (r.to.vect). Within the loop the “focal hole” is extracted based on category (v.extract) and the region is adjusted to this single polygon. By doing so, the r.mapcalc function for raster based region cropping could be replaced by the (significantly faster v.extract). Finally, if I am not mistaken the two runs of g.region could be joined to one (though this should not have major impact on the processing speed…).
Attachments (2)
Change History (15)
by , 11 years ago
| Attachment: | r.fillnulls.patch added |
|---|
comment:1 by , 11 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
comment:2 by , 10 years ago
| Component: | Shell Scripts → Python |
|---|---|
| CPU: | Unspecified → All |
| Milestone: | 7.0.0 → 7.1.0 |
| Platform: | Unspecified → All |
| Resolution: | fixed |
| Status: | closed → reopened |
Please find attached another proposal for speed-up of rst-interpolation by hole.
In the diff, there are two sections which can probably be removed in addition:
1) The new r.clump version should account for NoData, so the following r.mapcalc operation can be removed (could not test it since I work with an older version).
2) I was unsure if r.patch has a limit regarding the number of open maps, if not than block-wise application of r.patch can be removed too...
Maybe better to use bilinear or bicubic interpolation as default as it is significant faster (and simpler) than rst?
comment:5 by , 7 years ago
| Milestone: | 7.2.1 → 7.2.2 |
|---|
comment:6 by , 7 years ago
| Milestone: | 7.2.2 → 7.4.0 |
|---|
All enhancement tickets should be assigned to 7.4 milestone.
comment:9 by , 6 years ago
| Milestone: | 7.4.1 → 7.4.2 |
|---|
comment:10 by , 6 years ago
For a while r.fillnulls optionally uses r.resamp.bspline (#1088), anything left here?
comment:11 by , 6 years ago
| Milestone: | 7.4.2 → 7.6.0 |
|---|
All enhancement tickets should be assigned to 7.6 milestone.
A slightly modified version has been applied in r57030
Thanks, Stefan!
Next speed up for r.fillnulls would be if someone would implement parallel filling of holes.