v.surf.icw - not working with the last version of grass7

[bhlevca]

I updated my grass7 not thinking about compatibility issues. Several scripts used by v.surf.icw have changed among which g.remove

For example line 322

grass.run_command('g.remove', flags = 'f', rast = cost_site_name, quiet = True)

must be changed to.

 grass.run_command('g.remove', type='raster', flags = 'f', name = cost_site_name, quiet = True)

Now the module works partially but I struggled to find the final solution and I was wondering if someone who knows the code better can fix it.

I get the following errors after the successful creation of the : temporary files (tmp_icw*):

Creating partial weights ...
Site 1 of 51,  cat = 1, data value = 6.281
Site 2 of 51,  cat = 2, data value = 6.378843
Site 3 of 51,  cat = 3, data value = 6.324808
ERROR: Error reading raster data for row 281 of <MASK>
ERROR: Error reading raster data for row 2343 of <MASK>
ERROR: Error reading raster data for row 2109 of <MASK>

and at the end probably due to the former errors:

Calculating final values ...
Reading raster map <tmp_icw_25595_partial.00001> using weight 1.000000...
ERROR: Raster map <tmp_icw_25595_partial.00001> not found
ERROR: Problem running r.series
Cleanup..

This worked fine in grass 7 several months ago.

[bhlevca]

Sorry I meant v.surf.icw.

I tried to modify the subject but I could not figure out how.

[bhlevca]

I found the problem I had a dangling raster map named MASK that was screwing up.

I deleted that and now it works properly.

I am attaching my version and hopefully someone with access will update svn

Thanks

[annakrat]

I just fixed the g.remove calls in r64399.

[bhlevca]

modified v.surf.icw that works with grass7 trunk

[bhlevca]

I have a large number of points (~20000) and the script runs fine until it reaches the point of summation of cost weights.

Then Python gives up because there are too many items in the arguments list. I am wondering if the script could be changed to work around Python's limitation

Here is the error:

Removing anomalies at site positions ... Applying radial decay ... Finding sum of squares ... Summation of cost weights ... ... omitted lines ...

File "/usr/lib64/python2.7/subprocess.py", line 1327, in

_execute_child

raise child_exception

OSError: [Errno 7] Argument list too long

[lucadelu]

Replying to bhlevca:

I have a large number of points (~20000) and the script runs fine until it reaches the point of summation of cost weights.

Then Python gives up because there are too many items in the arguments list. I am wondering if the script could be changed to work around Python's limitation

Here is the error:

Removing anomalies at site positions ... Applying radial decay ... Finding sum of squares ... Summation of cost weights ... ... omitted lines ...

File "/usr/lib64/python2.7/subprocess.py", line 1327, in

_execute_child

raise child_exception

OSError: [Errno 7] Argument list too long

Could you please send us a command line to test it?

Possibly with north carolina dataset.

Thanks Luca

[bhlevca]

Sorry I don't have the North Carolina dataset but I can explain the command line I used below.

 v.surf.icw input=TorontoHarbourBathymetryWaterSimplifyedPoints column=Contour output=TorontoHarbourBathyInterpRast cost_map=TorontoHarbourWaterMaskCorrRast friction=3.5 --verbose workers=3

TorontoHarbourBathymetryWaterSimplifyedPoints has ~24000 points

The values attribute is Contour

The mask and the costmap is TorontoHarbourWaterMaskCorrRast just a plain polygons converted into a raster.

The problem is that when putting all the temporary maps together on the Python subprocess call it exceeds the maximum allowable size handled by Python or by the OS and I was wondering if there is another way of sequentially doing this.

see ​http://www.in-ulm.de/~mascheck/various/argmax/

[neteler]

Milestone renamed

[mlennert]

AFAIU, this comes from the r.series calls which use the input parameter with a list of maps. The solution is to use the file parameter instead, i.e. to write all map names to a file and to use that as input.

I'm attaching an untested diff which does this for the first r.series call. It might be necessary to the same for other r.series calls.

Please try with this patch to see if this is the right direction towards a solution.

[mlennert]

Could someone else test the proposed patch ?

[annakrat]

In 69618:

v.surf.icw: write temporary maps into file instead of passing through command line (by mlennert), see #2574

[hamish]

Replying to bhlevca:

I have a large number of points (~20000) and the script runs fine until it reaches the point of summation of cost weights.

Hi,

I'm just curious how this job went? The module was originally written for labor intensive water sample data (CTD) with maybe a couple hundred vector points max. Running r.cost 20000 times can be quite computationally intensive!

My usual approach for making a bathy surface interpolation -after r.in.xyz gridding is to use the r.surf.nnbathy module which makes a simple first pass mask (very similar to r.hull) then also run r.buffer on the input points at an appropriate mean distance between points *3 or so and use that as a better mask. v.to.rast area conversion of the coastline helps clip it too.

happy 2017, Hamish