Opened 4 years ago

Closed 13 months ago

Last modified 13 months ago

#5716 closed enhancement (fixed)

[PATCH] faster gdal_rasterize for a large number of small geometries

Reported by: youri Owned by: warmerdam
Priority: normal Milestone: 2.3.0
Component: Algorithms Version: svn-trunk
Severity: normal Keywords: faster, rasterize
Cc:

Description (last modified by youri)

Big performances enhancement for large number of small polygons.

The main idea is to loop over the geometries instead to loop over the pixels (by the way skipping out-of-scope geometries), and to write only concerned raster blocks. Very efficient for large files compared to original method. The old algorithm can theoretically still be more efficient for very large geometries (to be checked).

The attached files are working on svn-trunk rev 33442.

The default behaviour is to use the original algorithm when nb features < 1000, then use the new one. The algorithm can be explicitly set with the '-optim' param.

I hope this will help ! (some processes simply crash without this mod)

Attachments (5)

gdal_rasterize.cpp (36.4 KB) - added by youri 4 years ago.
gdalrasterize.cpp (61.8 KB) - added by youri 2 years ago.
gdal_rasterize_bin.cpp (7.1 KB) - added by youri 2 years ago.
gdal_rasterize_lib.cpp (44.6 KB) - added by youri 2 years ago.
gdal_alg_priv.h (8.3 KB) - added by youri 2 years ago.

Download all attachments as: .zip

Change History (13)

Changed 4 years ago by youri

Attachment: gdal_rasterize.cpp added

comment:1 Changed 4 years ago by Even Rouault

Summary: faster gdal_rasterize for a large number of small geometries[PATCH] faster gdal_rasterize for a large number of small geometries

comment:2 Changed 3 years ago by ctl101

Hi, Might you be able to provide a compiled replacement gdal_rasterize.py file with which those with no programming knowledge can make use of your enhancements? Best regards.

Changed 2 years ago by youri

Attachment: gdalrasterize.cpp added

Changed 2 years ago by youri

Attachment: gdal_rasterize_bin.cpp added

Changed 2 years ago by youri

Attachment: gdal_rasterize_lib.cpp added

Changed 2 years ago by youri

Attachment: gdal_alg_priv.h added

comment:3 Changed 2 years ago by youri

Description: modified (diff)
Version: unspecifiedsvn-trunk

comment:4 Changed 13 months ago by youri

Hello,

as it is quite time-consuming to adapt this mod for each version of gdal, is it possible to integrate it in the next stable version?

as written before, some applications just crash without it, and most-if-not-all runs see significant gain in processing time.

Thank you in advance !

comment:5 Changed 13 months ago by Even Rouault

Could you submit it possibly as a pull request over the github mirror at https://github.com/OSGeo/gdal ? This would help reviewing your changes and make sure they comply with existing enforced coding rulesand don't cause known regressions. Adding a new test case in the gdalautotest suite (e.g in autotest/alg/rasterize.py) to test your code would be needed too.

comment:6 Changed 13 months ago by youri

I just submitted a pull request. However, it is quite complex to add a test on speed as it concern large datasets (and quite large computing time for an autotest). We can however consider a test to ensure that the output results are similar.

comment:7 Changed 13 months ago by Even Rouault

Resolution: fixed
Status: newclosed

In 39204:

gdal_rasterize / GDALRasterizeGeometries(): optimize rasterization for large number of small geometries. Patch by youri / demaet. Fixes #5716 / https://github.com/OSGeo/gdal/pull/222

comment:8 Changed 13 months ago by Even Rouault

Milestone: 2.3.0
Note: See TracTickets for help on using tickets.