Opened 8 years ago

Closed 5 years ago

Last modified 5 years ago

#5716 closed enhancement (fixed)

[PATCH] faster gdal_rasterize for a large number of small geometries

Reported by: youri Owned by: warmerdam
Priority: normal Milestone: 2.3.0
Component: Algorithms Version: svn-trunk
Severity: normal Keywords: faster, rasterize
Cc:

Description (last modified by youri)

Big performances enhancement for large number of small polygons.

The main idea is to loop over the geometries instead to loop over the pixels (by the way skipping out-of-scope geometries), and to write only concerned raster blocks. Very efficient for large files compared to original method. The old algorithm can theoretically still be more efficient for very large geometries (to be checked).

The attached files are working on svn-trunk rev 33442.

The default behaviour is to use the original algorithm when nb features < 1000, then use the new one. The algorithm can be explicitly set with the '-optim' param.

I hope this will help ! (some processes simply crash without this mod)

Attachments (5)

gdal_rasterize.cpp (36.4 KB ) - added by youri 8 years ago.
gdalrasterize.cpp (61.8 KB ) - added by youri 6 years ago.
gdal_rasterize_bin.cpp (7.1 KB ) - added by youri 6 years ago.
gdal_rasterize_lib.cpp (44.6 KB ) - added by youri 6 years ago.
gdal_alg_priv.h (8.3 KB ) - added by youri 6 years ago.

Download all attachments as: .zip

Change History (13)

by youri, 8 years ago

Attachment: gdal_rasterize.cpp added

comment:1 by Even Rouault, 8 years ago

Summary: faster gdal_rasterize for a large number of small geometries[PATCH] faster gdal_rasterize for a large number of small geometries

comment:2 by ctl101, 7 years ago

Hi, Might you be able to provide a compiled replacement gdal_rasterize.py file with which those with no programming knowledge can make use of your enhancements? Best regards.

by youri, 6 years ago

Attachment: gdalrasterize.cpp added

by youri, 6 years ago

Attachment: gdal_rasterize_bin.cpp added

by youri, 6 years ago

Attachment: gdal_rasterize_lib.cpp added

by youri, 6 years ago

Attachment: gdal_alg_priv.h added

comment:3 by youri, 6 years ago

Description: modified (diff)
Version: unspecifiedsvn-trunk

comment:4 by youri, 5 years ago

Hello,

as it is quite time-consuming to adapt this mod for each version of gdal, is it possible to integrate it in the next stable version?

as written before, some applications just crash without it, and most-if-not-all runs see significant gain in processing time.

Thank you in advance !

comment:5 by Even Rouault, 5 years ago

Could you submit it possibly as a pull request over the github mirror at https://github.com/OSGeo/gdal ? This would help reviewing your changes and make sure they comply with existing enforced coding rulesand don't cause known regressions. Adding a new test case in the gdalautotest suite (e.g in autotest/alg/rasterize.py) to test your code would be needed too.

comment:6 by youri, 5 years ago

I just submitted a pull request. However, it is quite complex to add a test on speed as it concern large datasets (and quite large computing time for an autotest). We can however consider a test to ensure that the output results are similar.

comment:7 by Even Rouault, 5 years ago

Resolution: fixed
Status: newclosed

In 39204:

gdal_rasterize / GDALRasterizeGeometries(): optimize rasterization for large number of small geometries. Patch by youri / demaet. Fixes #5716 / https://github.com/OSGeo/gdal/pull/222

comment:8 by Even Rouault, 5 years ago

Milestone: 2.3.0
Note: See TracTickets for help on using tickets.