Opened 7 years ago

Closed 6 years ago

#3523 closed enhancement (fixed)

v.rast.stats: avoid multiple rasterization when run on multiple raster maps

Reported by: sbl Owned by: grass-dev@…
Priority: normal Milestone: 7.6.0
Component: Vector Version: svn-trunk
Keywords: v.rast.stats Cc:
CPU: All Platform: All

Description

When computing statistics from multiple raster maps (say terrain model, slope, aspect, ...) for vector geometries, v.rast.stats converts the input vector map in every module call. That generates significant though unnecessary overhead.

The simplest solution would be to allow multiple raster input (and multiple column prefixes).

Related ml-post: ​https://lists.osgeo.org/pipermail/grass-dev/2018-January/087004.html

I would work on it during community sprint.

Attachments (3)

v.rast.stats_multiple_rasters.py​ (12.9 KB ) - added by sbl 7 years ago.
Allow multiple input
v.rast.stats_multiraster.diff​ (13.2 KB ) - added by sbl 6 years ago.
Allow multiple raster input (diff updated to r73190)
test_patch_v.rast.stats_multiraster.png​ (113.1 KB ) - added by veroandreo 6 years ago.
test for patch allowing the use of multiple raster maps as input in v.rast.stats plus the new color tables :)

Download all attachments as: .zip

Change History (12)

comment:1 by martinl, 7 years ago

Milestone: → 7.6.0

by sbl, 7 years ago

Attachment: v.rast.stats_multiple_rasters.py​ added

Allow multiple input

comment:2 by sbl, 7 years ago

Attached a first attempt. Testing, feedback, comments welcome. Update of manual and test would be a next step...

comment:4 by sbl, 7 years ago

One thing I would like to get a second opinion on is how to handle the region adjustment that v.rast.stats currently performs. Originally, v.rast.stats aligns region temporarily to the input raster map (and does not alter extent), which of course becomes more tricky with multiple inputs.

In the attached patch the region is (for the time being) aligned to the first raster. Options for more proper handling would be to:
1) leave the current region untouched, so the user can choose resolution deliberately (could also become a new flag) 2) group raster maps by pixel alignment and perform conversion from vector to raster for every raster group (if raster maps differ in these regards) 3) as 2) but set extent from vector map in addition

Suggestions?

comment:5 by neteler, 6 years ago

Did anyone test the patch provided by sbl?

by sbl, 6 years ago

Attachment: v.rast.stats_multiraster.diff​ added

Allow multiple raster input (diff updated to r73190)

by veroandreo, 6 years ago

Attachment: test_patch_v.rast.stats_multiraster.png​ added

test for patch allowing the use of multiple raster maps as input in v.rast.stats plus the new color tables :)

comment:6 by veroandreo, 6 years ago

Tested with: 

g.copy vector=zipcodes,my_zipcodes
g.region -p vector=my_zipcodes align=elevation
v.rast.stats map=my_zipcodes raster=elevation,geology,soils column_prefix=elev,geo,soil method=average

I get a warning 

WARNING: Not all vector categories converted to raster. Converted 44 of 48.
Processing input data (44 categories)...
 100%
Updating the database ...
Processing input data (44 categories)...
 100%
Updating the database ...
Processing input data (44 categories)...
 100%
Updating the database ...

and then I only see data for 13 polygons, instead of 44 as the warning says... 

v.db.select my_zipcodes

cat|OBJECTID|WAKE_ZIPCO|PERIMETER|ZIPCODE_|ZIPCODE_ID|ZIPNAME|ZIPNUM|ZIPCODE|NAME|SHAPE_Leng|SHAPE_Area|elev_average|geo_average|soil_average
1|1|262493461.286|80746.40038|2|27|CREEDMOOR|27522|CREEDMOOR 27522|CREEDMOOR|80777.650719|262372872.587|||
2|2|31634621.1616|31064.35816|7|25|YOUNGSVILLE|27596|YOUNGSVILLE 27596|YOUNGSVILLE|31064.35845|31634622.1247|||
3|3|543964359.138|115252.44443|8|20|RALEIGH|27615|RALEIGH 27615|RALEIGH|115252.443861|543964358.996|||
4|4|58736529.666|36836.75795|10|26|YOUNGSVILLE|27596|YOUNGSVILLE 27596|YOUNGSVILLE|36836.756996|58736530.8791|||
5|7|12788390.3938|32416.21157|15|157|DURHAM|27703|DURHAM 27703|DURHAM|30664.931929|12593277.3902|||
6|8|602469071.217|147112.36983|18|8|RALEIGH|27616|RALEIGH 27616|RALEIGH|147116.461058|602468668.316|||
7|11|402520600.739|128456.89917|23|35|RALEIGH|27612|RALEIGH 27612|RALEIGH|128260.956758|402143653.823|||
8|12|1319493933.22|252236.30916|24|9|WENDELL|27591|WENDELL 27591|WENDELL|252265.618847|1319493242|||
9|13|313832905.094|80405.41507|25|19|RALEIGH|27609|RALEIGH 27609|RALEIGH|80405.413149|313832904.304|||
10|14|632112227.725|154800.00031|28|15|RALEIGH|27604|RALEIGH 27604|RALEIGH|154801.611532|632113047.308|80.9428977864892|270|31412.1938642022
11|15|873704032.742|198415.75507|29|22|KNIGHTDALE|27545|KNIGHTDALE 27545|KNIGHTDALE|207888.12556|854234759.151|||
12|17|86718740.3594|47692.08248|34|1|RALEIGH|27608|RALEIGH 27608|RALEIGH|47692.081615|86718740.3587|98.4611553894709|266.422191905412|33249.6981809914
13|19|1174353723.71|250639.52409|36|23|RALEIGH|27610|RALEIGH 27610|RALEIGH|260100.365867|1193825267.82|86.3101705216014|270|35158.4413575247
14|21|25495827.4688|25606.12087|38|2|RALEIGH|27605|RALEIGH 27605|RALEIGH|25606.118579|25495829.7998|107.106468511325|267.676296671735|35061.007475925
15|23|48955772.0156|33100.96362|41|21|RALEIGH|27601|RALEIGH 27601|RALEIGH|33100.961138|48955771.7285|93.4536230171682|270|41372.1275182546
16|24|672983954.444|137789.66922|42|18|APEX|27502|APEX 27502|APEX|138402.016283|671745658.098|||
17|25|1096882081.34|195776.53228|43|24|GARNER|27529|GARNER 27529|GARNER|195766.214382|1096940045.19|99.5202463771186|270|32040.7337491259
18|26|785094011.315|125961.78982|45|4|NEW HILL|27562|NEW HILL 27562|NEW HILL|125964.20281|785117168.913|||
19|27|74573603.2444|53026.74139|46|76|CLAYTON|27520|CLAYTON 27520|CLAYTON|53026.741222|74573603.668|||
20|29|14013604.9607|20227.39179|50|146|FUQUAY VARINA|27526|FUQUAY VARINA 27526|FUQUAY VARINA|20236.211152|14013892.9812|||
21|30|10825001.4179|18741.09609|51|5|WILLOW SPRING|27592|WILLOW SPRING 27592|WILLOW SPRING|18774.411343|10826786.022|||
22|32|595065.85534|3225.47552|53|147|ANGIER|27501|ANGIER 27501|ANGIER|3221.281589|594306.673565|||
23|33|880659423.548|230219.75758|44|11|APEX|27539|APEX 27539|APEX|10758.608766|3709374.42305|||
24|36|607850854.951|135414.36225|6|149|RALEIGH|27613|RALEIGH 27613|RALEIGH|135135.145707|610078059.502|||
25|39|118820937.008|55979.83097|9|3|ROLESVILLE|27571|ROLESVILLE 27571|ROLESVILLE|60467.130446|128973370.856|||
26|40|821009204.045|192414.44366|5|12|RALEIGH|27614|RALEIGH 27614|RALEIGH|193326.052701|821170463.631|||
27|45|246107221.69|98230.55446|14|28|RALEIGH|27617|RALEIGH 27617|RALEIGH|96973.596277|242589284.931|||
28|54|1380544316.71|223529.01406|13|14|ZEBULON|27597|ZEBULON 27597|ZEBULON|229111.795818|1389077516.66|||
29|114|595719341.254|135382.61154|40|10|CARY|27518|CARY 27518|CARY|118402.346061|275997437.908|121.758819832926|489.922880820543|30329.1413058828
30|212|352353219.773|137980.26455|35|18|APEX|27523|APEX 27523|APEX|126164.348014|325402958.569|||
31|219|13347044.5484|17858.88115|52|147|ANGIER|27501|ANGIER 27501|ANGIER|20517.250079|16785459.3838|||
32|222|670507534.879|180573.69876|49|5|WILLOW SPRING|27592|WILLOW SPRING 27592|WILLOW SPRING|183229.723255|667072962.625|||
33|231|897980333.539|201783.43174|47|6|HOLLY SPRINGS|27540|HOLLY SPRINGS 27540|HOLLY SPRINGS|232065.063395|914275342.706|||
34|236|880659423.548|230219.75758|44|11|APEX|27539|APEX 27539|APEX|251643.611051|892646046.022|121.77684366211|542.152999645012|33788.8520992674
35|244|595719341.254|135382.61154|40|10|CARY|27511|CARY 27511|CARY|88289.696983|325536306.021|138.362605235083|450.144147549038|26780.3082788465
36|260|490635823.637|134946.66576|27|18|CARY|27519|CARY 27519|CARY|129180.587307|564062896.506|||
37|275|2825789004.85|465259.10126|3|13|WAKE FOREST|27587|WAKE FOREST 27587|WAKE FOREST|477264.995577|2806740621.72|||
38|276|504275401.693|157917.82636|30|18|CARY|27513|CARY 27513|CARY|164487.956602|509387886.317|142.599166816593|463.879361016306|21902.406996587
39|282|679964927.93|203429.0581|22|34|RALEIGH|27607|RALEIGH 27607|RALEIGH|182366.287225|552333204.751|129.716638610443|299.799134494058|23540.2665508821
40|286|1285870010.66|282815.79339|37|66|RALEIGH|27603|RALEIGH 27603|RALEIGH|285693.495599|1408742751.36|97.3470864982172|329.630266177917|30374.4847784233
41|288|526570086.386|173734.20912|19|33|MORRISVILLE|27560|MORRISVILLE 27560|MORRISVILLE|169015.912917|643660203.58|||
42|298|829874917.625|230773.26059|39|2|RALEIGH|27606|RALEIGH 27606|RALEIGH|212707.32257|679989401.948|117.274242541568|250.214032731026|29381.817335321
43|301|1073274000.75|252721.8937|48|7|FUQUAY VARINA|27526|FUQUAY VARINA 27526|FUQUAY VARINA|270514.968365|1061355345.52|||
44|196|44695436.7135|43678.90844|26|150|DURHAM|27713|DURHAM 27713|DURHAM|19049.668725|7833481.75676|||

which makes sense since all maps queried are much smaller than the region covered by the vector (See attachment). So, seems to work.

comment:7 by neteler, 6 years ago

Please commit the patch, otherwise it will get lost or outdated. Thanks

comment:8 by sbl, 6 years ago

In 73461:

apply multiraster patch see #3523

comment:9 by sbl, 6 years ago

Resolution: → fixed
Status: new → closed

In 73462:

apply multiraster patch; fix #3523

Note: See TracTickets for help on using tickets.