Increase maximum number of vector points available

[dnewcomb]

The current number of points in a imported into a vector data layer appears to be limited to a 32 bit integer ( 2 billion) . You are allowed to import more, but the number of points are indicated with a negative number Not sure if the data set is still valid at that point.

The primary use case for this is lidar data. The new USGS standard for landscape LiDAR data collection is QL2 , ​http://pubs.usgs.gov/tm/11b4/ , roughly 2 points/meter density. The lidar ground points at this density can exceed the 32 bit integer limit in a 1000 km^{2 area (31km x 31km). I already have counties in NC, USA with ground point data that exceed this threshold.}

Newer LiDAR technologies ( Single Photon, Geiger Mode) with collection densities of 20 points/meter are already available, but r.in.lidar may be more suitable for these data sets in the short term.

[neteler]

Replying to dnewcomb:

The current number of points in a imported into a vector data layer appears to be limited to a 32 bit integer ( 2 billion).

Isn't this the limit in case of topology being enabled? Did you try to disable topology during import?

[dnewcomb]

Topology was disabled. Attribute table was not built. Looking back, I am running an svn snapshot from 1-10-2015, was this addressed before the final release?

[neteler]

Replying to dnewcomb:

Topology was disabled. Attribute table was not built.

OK - this is an important information.

Looking back, I am running an svn snapshot from 1-10-2015, was this addressed before the final release?

No idea but there were definitely important vector fixes implemented after that date: ​https://trac.osgeo.org/grass/log/grass/trunk/lib/vector

[dnewcomb]

Will upgrade to latest weekly and report back

[dnewcomb]

Installed 4-4-15 weekly svn snapshot and started import of las file with roughly 2.4 billion points. v.in.lidar is reporting scanning -1.87 billion points on import.

[marisn]

Could you, please, copy / paste exact message text? It could be as simple as wrong formatting character used while printing message or overflowing counter used just to print progress.

[dnewcomb]

Replying to marisn:

Could you, please, copy / paste exact message text? It could be as simple as wrong formatting character used while printing message or overflowing counter used just to print progress.

v.in.lidar -t -o -b input=/media/Seagate Expansion Drive/BladenCoNC/bladen_ground0.laz output=bladen_ground_pts Over-riding projection check Scanning -1879965037 points...

When I ran v.in.lidar on the previous version of GRASS 7 that I was using, I ran v.info on the resulting vector data set and go the same number for the number of points .

Here is the info on the lidar file:

newcomb@IFW4FO-RAL-SGIS:/media/Seagate Expansion Drive/BladenCoNC$ ~/lastools/lastools/bin/lasinfo bladen_ground0.laz reporting all LAS header entries:

file signature: 'LASF' file source ID: 0 global_encoding: 1 project ID GUID data 1-4: 00000000-0000-0000-0000-000000000000 version major.minor: 1.3 system identifier: 'LAStools (c) by Martin Isenburg' generating software: 'lasmerge (version 131210)' file creation day/year: 252/2014 header size: 235 offset to point data: 345 number var. length records: 1 point data format: 3 point data record length: 34 number of point records: 2415002259 number of points by return: 1503313306 704621650 204714902 2216754 135647 scale factor x y z: 0.01 0.01 0.01 offset x y z: -0 -0 -0 min x y z: 2030000.00 225000.00 3.92 max x y z: 2244999.99 404999.99 174.24

[dnewcomb]

Replying to dnewcomb: Looking back, this seems to be a duplication of this ticket, which I forgotten I had submitted ​http://trac.osgeo.org/grass/ticket/2472

Replying to marisn:

Could you, please, copy / paste exact message text? It could be as simple as wrong formatting character used while printing message or overflowing counter used just to print progress.

v.in.lidar -t -o -b input=/media/Seagate Expansion Drive/BladenCoNC/bladen_ground0.laz output=bladen_ground_pts Over-riding projection check Scanning -1879965037 points...

When I ran v.in.lidar on the previous version of GRASS 7 that I was using, I ran v.info on the resulting vector data set and go the same number for the number of points .

Here is the info on the lidar file:

newcomb@IFW4FO-RAL-SGIS:/media/Seagate Expansion Drive/BladenCoNC$ ~/lastools/lastools/bin/lasinfo bladen_ground0.laz reporting all LAS header entries:

file signature: 'LASF' file source ID: 0 global_encoding: 1 project ID GUID data 1-4: 00000000-0000-0000-0000-000000000000 version major.minor: 1.3 system identifier: 'LAStools (c) by Martin Isenburg' generating software: 'lasmerge (version 131210)' file creation day/year: 252/2014 header size: 235 offset to point data: 345 number var. length records: 1 point data format: 3 point data record length: 34 number of point records: 2415002259 number of points by return: 1503313306 704621650 204714902 2216754 135647 scale factor x y z: 0.01 0.01 0.01 offset x y z: -0 -0 -0 min x y z: 2030000.00 225000.00 3.92 max x y z: 2244999.99 404999.99 174.

[wenzeslaus]

For the case of v.in.lidar this is a duplicate of #2472, closing.

[neteler]

Milestone renamed