v.kernel: suspected i/o bottleneck

[dylan]

v.kernel seems to have a disk i/o bottleneck, that starts to cause problems with moderately large input vector files. I use this command with <3000 point features, with a small region setting: approx 500x500 cells, regularly without issue. Trying out a larger dataset, 22k point features -- same region geometry, results in a lot of disk activity and very slow run-times. I took a peak at the source code, and it looks like there are several calls to a spatial index. There was some chatter on the mailing list a while back regarding improvement in the way that the spatial index is created. I wonder if this would improve v.kernel when working with a large number of features.

The reason that I suspect some kind of I/O issue -- I hear a lot of disk trashing while this module is running. Also note that it is using very little memory... perhaps some caching would speed things up.

Some details on the environment:

# region info:
rows:       632
cols:       364
cells:      230048

# command and arguments
# file 'soilweb_queries' has approx 22k records
v.kernel --o in=soilweb_queries out=swq std=50000

[hamish]

can you run it with "time"? ie:

time v.kernel --o in=soilweb_queries out=swq std=50000

that will say how much time is spent doing the work, and how much time is taken up by the kernel dealing i/o stuff, etc.

Hamish

[dylan]

Some data:

std,real,user,sys
100,1.621,1.516,0.100
500,1.854,1.612,0.244
1000,2.680,2.032,0.632
5000,25.090,12.993,12.085
8000,61.72,31.078,30.610
10000,97.877,48.631,49.163
50000,2640,,

Hmm... the sys times do not seem to dominate, rather they approach that of the usr time.

[neteler]

v.kernel has been made faster in r54108, r53983, r53982, r53956, r53896. See also trac #1800. Closing this report as fixed.