v.rast.stats on dbf shapefile not enough precision even with DOUBLE PRECISION

[vesnikos]

When I'm doing v.rast.stats with grass (v6.4.2) and a polygon with a dbf db, the results come back for each row with two decimental numbers. The column attribute is of type "DOUBLE PRECISION" . That presicion is not enough.

for grass7 the v.rast.stats come back with much better presicion(<6 decimental digits)

​http://i49.tinypic.com/2iqohmu.png

[neteler]

Please check the real precision with the output of

v.db.select yourmap

I suspect that the table visualization tool just doesn't show more digits which may effectively be there.

[hamish]

I expect that MarkusN's suspicion is correct, but for completeness see also #335 if doubt remains.

Hamish

[elcol]

I can confirm this bug. I also only get 2 decimals instead of double precision for column attributes, also when using v.db.select. I am using GRASS 6.4.2

Elco

[hamish]

Replying to elcol:

I can confirm this bug. I also only get 2 decimals instead of double precision for column attributes, also when using v.db.select. I am using GRASS 6.4.2

is the database backend dbf?

if so, can you look at it with a utility like 'dbview' or {Open|Libre}Office Calc and inspect the data values in it? Perhaps the raw data is that way and v.db.* is just rounding off the .000000000s

Hamish

[elcol]

Replying to hamish:

Replying to elcol:

I can confirm this bug. I also only get 2 decimals instead of double precision for column attributes, also when using v.db.select. I am using GRASS 6.4.2

is the database backend dbf?

if so, can you look at it with a utility like 'dbview' or {Open|Libre}Office Calc and inspect the data values in it? Perhaps the raw data is that way and v.db.* is just rounding off the .000000000s

Hamish

I'm using the default dbf backend. I've just checked the .dbf file with LibreOffice, and all data has only 2 decimals in the actual file too.

[hamish]

Replying to elcol:

I'm using the default dbf backend. I've just checked the .dbf file with LibreOffice, and all data has only 2 decimals in the actual file too.

ok, how was the data imported? was the original source like that too?

Hamish

[ElcoL]

Replying to hamish:

Replying to elcol:

I'm using the default dbf backend. I've just checked the .dbf file with LibreOffice, and all data has only 2 decimals in the actual file too.

ok, how was the data imported? was the original source like that too?

Hamish

Its slope data that i got using r.slope.aspect on a DEM. Map query reports up to 10 decimals, so the dataset looks normal.

Elco

[hamish]

Replying to ElcoL:

Its slope data that i got using r.slope.aspect on a DEM. Map query reports up to 10 decimals, so the dataset looks normal.

Which method did you use to convert the raster result to a vector map? i.e. exact command line arguments used with v.rast.stats? Did you use the -e flag? what was the vector polygon map like? does 'r.univar -t' look normal on the query map?

Is the normal looking map query of the DEM or the result of r.slope.aspect?

Hamish

[hamish]

.. and can you try with GRASS 6.4.3rc4 or a 6.4.3svn nightly snapshot? which Linux distro are you using?

Hamish

[ElcoL]

Replying to hamish:

.. and can you try with GRASS 6.4.3rc4 or a 6.4.3svn nightly snapshot? which Linux distro are you using?

Hamish

I was running GRASS 6.4.2 on Ubuntu 13.04, but I have just reproduced the error too after compiling the current svn snapshot (ie. GRASS 6.4.3svn).

Here's the exact command for v.rast.stats: v.rast.stats -c vector=na_pfaf_alpha@PERMANENT raster=watertable_gradient_fraction@PERMANENT colprefix=wtg

the watertable_gradient_fraction@PERMANENT raster is the result of r.slope.aspect and looks normal (ie more than 2 decimals). The resulting attribute table of na_pfaf_alpha@PERMANENT only contains 2 decimals in queries or when inspecting the .dbf file with gnumeric or libreoffice

[hamish]

can you also provide the r.slope.aspect command? (r.info -h).

and can you reproduce it with one of the sample datasets? (spearfish or NC, so I can try to reproduce it here too)

thanks, Hamish

ps- no need to explicitly state @PERMANENT, it's automatically in the map search path.

[ElcoL]

Replying to hamish:

can you also provide the r.slope.aspect command? (r.info -h).

and can you reproduce it with one of the sample datasets? (spearfish or NC, so I can try to reproduce it here too)

thanks, Hamish

ps- no need to explicitly state @PERMANENT, it's automatically in the map search path.

I reproduced the error with the NC dataset:

v.rast.stats vector=soils_wake@PERMANENT raster=slope@PERMANENT colprefix=slope

the result is an attribute table with a lot of empty rows and the rows that do show slope stats all have 2 decimals only

[hamish]

Replying to ElcoL:

I reproduced the error with the NC dataset:

ok, thanks, I can reproduce it now too.

(best to work in a user mapset, and not modify the maps in PERMANENT, make a copy and work on the copy)

It happens because the awk command specifically sets the print format to "%.2f".

​https://trac.osgeo.org/grass/browser/grass/branches/develbranch_6/scripts/v.rast.stats/v.rast.stats#L363

so you only get 2 digits after the decimal place.

seems this came in with r48058 as part of a bulk merge with another (faster) version of the script, so in 6.4.2+ but not the old+slow 6.4.1 version. I don't see a reason to keep it as %.2f instead of %.{something}g, any hints Otto?

Hamish

[neteler]

Replying to hamish: ...

It happens because the awk command specifically sets the print format to "%.2f".

...

seems this came in with r48058 as part of a bulk merge ...

I don't see a reason to keep it as %.2f instead of %.{something}g, any hints Otto?

To which precision should we change this?

[neteler]

In r57560 it got already changed to %.15g, so that looks like solved.

Test case (long line broken by me for trac formatting):

v.db.connect mysoils_wake -p
g.copy vect=soils_wake,mysoils_wake
v.rast.stats vector=mysoils_wake raster=slope colprefix=slope
v.db.select mysoils_wake
...
3364|120028|1653.86|35468|34409|Co|B|34|0.142551511526108|2.868412733078|
  2.7258612215519|1.10701639950275|0.692389694214987|0.479403488655124|
  62.5455679361202|37.6385575830936
...

d.mon x0
d.rast slope 
d.vect type=boundary

To speed v.rast.stats up I added DB TRANSACTION in r60243, r60244.

Closing as solved.