Opened 16 years ago
Closed 12 years ago
#567 closed enhancement (fixed)
v.kernel: suspected i/o bottleneck
Reported by: | dylan | Owned by: | |
---|---|---|---|
Priority: | normal | Milestone: | 6.5.0 |
Component: | Default | Version: | svn-develbranch6 |
Keywords: | v.kernel | Cc: | |
CPU: | x86-32 | Platform: | Linux |
Description
v.kernel seems to have a disk i/o bottleneck, that starts to cause problems with moderately large input vector files. I use this command with <3000 point features, with a small region setting: approx 500x500 cells, regularly without issue. Trying out a larger dataset, 22k point features -- same region geometry, results in a lot of disk activity and very slow run-times. I took a peak at the source code, and it looks like there are several calls to a spatial index. There was some chatter on the mailing list a while back regarding improvement in the way that the spatial index is created. I wonder if this would improve v.kernel when working with a large number of features.
The reason that I suspect some kind of I/O issue -- I hear a lot of disk trashing while this module is running. Also note that it is using very little memory... perhaps some caching would speed things up.
Some details on the environment:
# region info: rows: 632 cols: 364 cells: 230048 # command and arguments # file 'soilweb_queries' has approx 22k records v.kernel --o in=soilweb_queries out=swq std=50000
Change History (3)
comment:1 by , 16 years ago
comment:2 by , 16 years ago
Some data:
std,real,user,sys 100,1.621,1.516,0.100 500,1.854,1.612,0.244 1000,2.680,2.032,0.632 5000,25.090,12.993,12.085 8000,61.72,31.078,30.610 10000,97.877,48.631,49.163 50000,2640,,
Hmm... the sys times do not seem to dominate, rather they approach that of the usr time.
can you run it with "time"? ie:
that will say how much time is spent doing the work, and how much time is taken up by the kernel dealing i/o stuff, etc.
Hamish