Opened 4 years ago

Closed 3 years ago

Last modified 3 years ago

#2651 closed enhancement (duplicate)

Increase maximum number of vector points available

Reported by: dnewcomb Owned by: grass-dev@…
Priority: normal Milestone: 7.2.0
Component: Vector Version: svn-trunk
Keywords: v.in.lidar, lidar, vector point data Cc:
CPU: Unspecified Platform: Unspecified

Description

The current number of points in a imported into a vector data layer appears to be limited to a 32 bit integer ( 2 billion) . You are allowed to import more, but the number of points are indicated with a negative number Not sure if the data set is still valid at that point.

The primary use case for this is lidar data. The new USGS standard for landscape LiDAR data collection is QL2 , http://pubs.usgs.gov/tm/11b4/ , roughly 2 points/meter density. The lidar ground points at this density can exceed the 32 bit integer limit in a 1000 km2 area (31km x 31km). I already have counties in NC, USA with ground point data that exceed this threshold.

Newer LiDAR technologies ( Single Photon, Geiger Mode) with collection densities of 20 points/meter are already available, but r.in.lidar may be more suitable for these data sets in the short term.

Change History (10)

comment:1 in reply to:  description Changed 4 years ago by neteler

Keywords: lidar added
Version: unspecifiedsvn-trunk

Replying to dnewcomb:

The current number of points in a imported into a vector data layer appears to be limited to a 32 bit integer ( 2 billion).

Isn't this the limit in case of topology being enabled? Did you try to disable topology during import?

comment:2 Changed 4 years ago by dnewcomb

Topology was disabled. Attribute table was not built. Looking back, I am running an svn snapshot from 1-10-2015, was this addressed before the final release?

comment:3 in reply to:  2 Changed 4 years ago by neteler

Replying to dnewcomb:

Topology was disabled. Attribute table was not built.

OK - this is an important information.

Looking back, I am running an svn snapshot from 1-10-2015, was this addressed before the final release?

No idea but there were definitely important vector fixes implemented after that date: https://trac.osgeo.org/grass/log/grass/trunk/lib/vector

comment:4 Changed 4 years ago by dnewcomb

Will upgrade to latest weekly and report back

comment:5 Changed 4 years ago by dnewcomb

Installed 4-4-15 weekly svn snapshot and started import of las file with roughly 2.4 billion points. v.in.lidar is reporting scanning -1.87 billion points on import.

comment:6 Changed 4 years ago by marisn

Could you, please, copy / paste exact message text? It could be as simple as wrong formatting character used while printing message or overflowing counter used just to print progress.

comment:7 in reply to:  6 ; Changed 4 years ago by dnewcomb

Replying to marisn:

Could you, please, copy / paste exact message text? It could be as simple as wrong formatting character used while printing message or overflowing counter used just to print progress.

v.in.lidar -t -o -b input=/media/Seagate Expansion Drive/BladenCoNC/bladen_ground0.laz output=bladen_ground_pts Over-riding projection check Scanning -1879965037 points...

When I ran v.in.lidar on the previous version of GRASS 7 that I was using, I ran v.info on the resulting vector data set and go the same number for the number of points .

Here is the info on the lidar file:

newcomb@IFW4FO-RAL-SGIS:/media/Seagate Expansion Drive/BladenCoNC$ ~/lastools/lastools/bin/lasinfo bladen_ground0.laz reporting all LAS header entries:

file signature: 'LASF' file source ID: 0 global_encoding: 1 project ID GUID data 1-4: 00000000-0000-0000-0000-000000000000 version major.minor: 1.3 system identifier: 'LAStools (c) by Martin Isenburg' generating software: 'lasmerge (version 131210)' file creation day/year: 252/2014 header size: 235 offset to point data: 345 number var. length records: 1 point data format: 3 point data record length: 34 number of point records: 2415002259 number of points by return: 1503313306 704621650 204714902 2216754 135647 scale factor x y z: 0.01 0.01 0.01 offset x y z: -0 -0 -0 min x y z: 2030000.00 225000.00 3.92 max x y z: 2244999.99 404999.99 174.24

comment:8 in reply to:  7 ; Changed 4 years ago by dnewcomb

Replying to dnewcomb: Looking back, this seems to be a duplication of this ticket, which I forgotten I had submitted http://trac.osgeo.org/grass/ticket/2472

Replying to marisn:

Could you, please, copy / paste exact message text? It could be as simple as wrong formatting character used while printing message or overflowing counter used just to print progress.

v.in.lidar -t -o -b input=/media/Seagate Expansion Drive/BladenCoNC/bladen_ground0.laz output=bladen_ground_pts Over-riding projection check Scanning -1879965037 points...

When I ran v.in.lidar on the previous version of GRASS 7 that I was using, I ran v.info on the resulting vector data set and go the same number for the number of points .

Here is the info on the lidar file:

newcomb@IFW4FO-RAL-SGIS:/media/Seagate Expansion Drive/BladenCoNC$ ~/lastools/lastools/bin/lasinfo bladen_ground0.laz reporting all LAS header entries:

file signature: 'LASF' file source ID: 0 global_encoding: 1 project ID GUID data 1-4: 00000000-0000-0000-0000-000000000000 version major.minor: 1.3 system identifier: 'LAStools (c) by Martin Isenburg' generating software: 'lasmerge (version 131210)' file creation day/year: 252/2014 header size: 235 offset to point data: 345 number var. length records: 1 point data format: 3 point data record length: 34 number of point records: 2415002259 number of points by return: 1503313306 704621650 204714902 2216754 135647 scale factor x y z: 0.01 0.01 0.01 offset x y z: -0 -0 -0 min x y z: 2030000.00 225000.00 3.92 max x y z: 2244999.99 404999.99 174.

comment:9 in reply to:  8 Changed 3 years ago by wenzeslaus

Keywords: v.in.lidar added
Resolution: duplicate
Status: newclosed

For the case of v.in.lidar this is a duplicate of #2472, closing.

comment:10 Changed 3 years ago by neteler

Milestone: 7.1.07.2.0

Milestone renamed

Note: See TracTickets for help on using tickets.