Painfully Slow 'v.in.ogr' Vector Import
|Reported by:||justinzane||Owned by:|
|Keywords:||import, OGR, performance, v.in.ogr||Cc:||justinzane|
I'm trying to import shapefiles (in this case, OpenStreetMap data) into GRASS (grass70-svn) using
v.in.ogr; and it is painfully, brutally slow. By slow, I mean between 13 and 28 primitives per second.
At first I thought this was because I was using the default sqlite database on an NFS mount. But, trying sqlite on a local disk and trying MySQL to separate server both failed to improve performance. I'm getting a consistent read/write of ~125Kbps/~550Kbps no matter which shapefile or which database I use.
I cannot see any bottlenecks anywhere, as the two
python /path/to/grass-gui processes are averaging 10% of one core combined, the network utilization is below 10% consistently, and the DB server is almost quiescent. To give an example of how painful this is:
v.in.ogr dsn=/home/justin/downloads/osm_CA/places.shp WARNING: All available OGR layers will be imported into vector map <places> Check if OGR layer <places> contains polygons... Importing 7020 features (OGR layer <places>)... ----------------------------------------------------- Building topology for vector map <places@OSM>... Registering primitives... 7020 primitives registered 7020 vertices registered Building areas... 0 areas built 0 isles built Attaching islands... Attaching centroids... Number of nodes: 0 Number of primitives: 7020 Number of points: 7020 Number of lines: 0 Number of boundaries: 0 Number of centroids: 0 Number of areas: 0 Number of isles: 0 (Fri Jan 31 19:01:08 2014) Command finished (4 min 38 sec)
Note that this may belong under Database.