Large attribute table causes buffer overflow in v.db.select

[marisn]

wxGUI attribute table window is trying to load all values from the attribute table. In case of large number of values, it is causing an overflow in v.db.select. wxGUI should not attempt to load more than N rows at the time (where 100 > N >= 1000).

Tested with: Number of centroids: 99249

*** buffer overflow detected ***: v.db.select terminated
======= Backtrace: =========
/lib64/libc.so.6(+0x6fb0c)[0x7f542c5a7b0c]
/lib64/libc.so.6(__fortify_fail+0x37)[0x7f542c62fe27]
/lib64/libc.so.6(+0xf5f20)[0x7f542c62df20]
/lib64/libc.so.6(+0xf53f9)[0x7f542c62d3f9]
/lib64/libc.so.6(_IO_default_xsputn+0x80)[0x7f542c5ab270]
/lib64/libc.so.6(_IO_vfprintf+0x13a2)[0x7f542c57fc62]
/lib64/libc.so.6(__vsprintf_chk+0x8c)[0x7f542c62d48c]
/lib64/libc.so.6(__sprintf_chk+0x7d)[0x7f542c62d3dd]
v.db.select(main+0x4ad)[0x401d4d]
/lib64/libc.so.6(__libc_start_main+0xf0)[0x7f542c558710]
v.db.select(_start+0x29)[0x4025f9]
======= Memory map: ========
00400000-00403000 r-xp 00000000 08:05 12518553                           /home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/bin/v.db.select
00602000-00603000 r--p 00002000 08:05 12518553                           /home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/bin/v.db.select
00603000-00604000 rw-p 00003000 08:05 12518553                           /home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/bin/v.db.select

[annakrat]

Aren't these 2 separate issues? First is v.db.select crashing and second is opening large tables in attribute table manager.

[marisn]

Replying to annakrat:

Aren't these 2 separate issues? First is v.db.select crashing and second is opening large tables in attribute table manager.

I haven't been reading the code, still I believe the overflow is caused by behaviour of wxGUI as it happened when I was trying to view attribute table of vector map. The question is if we should fix v.db.selec as many parts of GRASS would fail if abused (just grep for fixed buffer sizes in C files to get an idea).

[annakrat]

Replying to marisn:

Replying to annakrat:

Aren't these 2 separate issues? First is v.db.select crashing and second is opening large tables in attribute table manager.

I haven't been reading the code, still I believe the overflow is caused by behaviour of wxGUI as it happened when I was trying to view attribute table of vector map. The question is if we should fix v.db.selec as many parts of GRASS would fail if abused (just grep for fixed buffer sizes in C files to get an idea).

The backtrace comes from v.db.select, so we should start there. Loading data in attribute manager is separate issue. Run v.db.select from command line, you should get the backtrace. But to fix it we need reproducible example, so if you can't reproduce it on North Carolina, your data is needed or at least a way to generate the data which lead to the crash.

[marisn]

Sorry, Anna, I was partially wrong. After examining dbmgr.base VirtualAttributeList with lots of print's, I came down to line joining all column names to pass them to v.db.select. (​Line 120 of dbmgr/base.py) Thus it was causing an overflow in select statement of v.db.select and I could not see it from CLI, as I was running it without explicitly stating all columns. Once I saw exact command executed by wxGUI, fixing it was easy.

I bumped up string buffer for storing query from 1024 to 4096 in r68321 (backported to 7.0 in r68322). Should be enough for most of cases as "640k should be enough for everyone".