v.colors freezes using the pg driver and attr

[jamesp670]

Hi Grass Devs,

You've got a big fan in me. I acknowledged GRASS on my AGU (see agu.org) poster this year and I'll make sure it's a cite next year.

I have a bug report. This is getting complex and perhaps you can help?

The setup:

My uname output:

: GRASS 7.0.0svn (rlis-master-for-seven):~ > uname -a
: Linux login.cluster 2.6.32-431.17.1.el6.x86_64 #1 SMP Wed May 7 23:32:49 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

I built grass from source, using primarily deps supplied by

   : yum install http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/\
   : pgdg-redhat93-9.3-1.noarch.rpm

The build is painless. GRASS works great, except for this bug.

I run GRASS v.colors to obtain a color map for TIGER census, as in:

: GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose map=tract2010censusdp1_clipped_02 column=popden10 layer=1 rgb_column=GRASSRGB color=grey

What I expect:

GRASSRGB column is created and populated with greyscale values.

What happens instead, is that v.colors freezes. Specifically, I see this sequence:

: GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose map=tract2010censusdp1_clipped_02 column=popden10 layer=1 rgb_column=GRASSRGB color=grey
: Option <column> given, assuming <use=attr>...
: Converting color rules into categories...
: Writing color rules...
: DBMI-PostgreSQL driver error:
: Unable to execute:
: ALTER TABLE tract2010censusdp1_clipped_02 ADD COLUMN GRASSRGB VARCHAR(11)
: ERROR:  column "grassrgb" of relation "tract2010censusdp1_clipped_02" already exists
: 
: 
: DBMI-PostgreSQL driver error:
: Unable to execute:
: ALTER TABLE tract2010censusdp1_clipped_02 ADD COLUMN GRASSRGB VARCHAR(11)
: ERROR:  column "grassrgb" of relation "tract2010censusdp1_clipped_02" already exists
: 
: 
: ERROR: Unable to add column <GRASSRGB> to table
:        <tract2010censusdp1_clipped_02>
: no database is open
: no database is open
:   C-c C-c
: GRASS 7.0.0svn (rlis-master-for-seven):~ > 

Where the C-c C-c is, I am tapping C-c to break the task, which is blatently frozen. I've debugged it. The sequence of events is:

: print "no database is open"
: db_shutdown_driver(driver=00661C40)
: doing waitpid(pid=1583)
: [freeze]

During the freeze, proc 1583 is still running, and it's a db/pg process. If I "kill 1583", the v.colors job ends immediately with no additional output, as in

: no database is open
: no database is open
: [I kill the db/pg process by hand]
: GRASS 7.0.0svn (rlis-master-for-seven):~ > 

I've tried v.colors in the trunk and release SVN branches. The layer is fine, I can query it, view attributes just fine. db.test output looks fine. It's 100% reproducable.

I build from source, debug with gdb, insert print statements. That's what got me this far, but now this IPC with the grass-7.0.0svn/driver/db/pg process is getting pretty complex. If I had to hazard a guess, I'd say that closing the communication FILEs in shutdown.c should cause the db/pg process to end. It is not for some reason. That then also begs the question of why the shutdown is called before the colors are written at all. The stack is:

: (gdb) where
: ,#0  0x0000003ab72ac8ce in waitpid () from /lib64/libc.so.6
: ,#1  0x00002aaaab17f3ad in G_wait (i_pid=28458) at spawn.c:981
: ,#2  0x00002aaaaad37348 in db_shutdown_driver (driver=0x661ec0) at shutdown.c:59
: ,#3  0x00002aaaaad35e60 in error_handler_driver (p=0x661ec0) at handler.c:24
: ,#4  0x00002aaaab1630ad in G__call_error_handlers () at handler.c:108
: ,#5  0x00002aaaab15ee93 in G_fatal_error (
:     msg=0x405368 "Unable to add column <%s> to table <%s>") at error.c:176
: ,#6  0x00000000004042b1 in write_rgb_values (Map=0x7fffffffcfb0, layer=1, 
:     column_name=0x654830 "GRASSRGB", colors=0x7fffffffce90) at write_rgb.c:37
: ,#7  0x0000000000402f83 in main (argc=7, argv=0x7fffffffd988) at main.c:350

If I delete grassrgb by hand, v.colors does return neatly and works great, as in:

: psql (9.3.5)
: => 
: => alter table tract2010censusdp1_clipped_02 drop column grassrgb;
: ALTER TABLE
: => 

then

: GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose map=tract2010censusdp1_clipped_02 column=popden10 layer=1 rgb_column=GRASSRGB color=grey
h: Option <column> given, assuming <use=attr>...
: Converting color rules into categories...
: Writing color rules...
: Column <GRASSRGB> added to table <tract2010censusdp1_clipped_02>
: 
: Color table for vector map <tract2010censusdp1_clipped_02@PERMANENT> set to
: 'grey'
: GRASS 7.0.0svn (rlis-master-for-seven):~ > 

The content of GRASSRGB looks fine after that.

Removing the color table using -r before running doesn't work, I get

: GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors -r map=tract2010censusdp1_clipped_02 rgb_column=GRASSRGB
: WARNING: Color table of vector map <tract2010censusdp1_clipped_02> not
:          found
: GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose map=tract2010censusdp1_clipped_02 column=popden10 layer=1 rgb_column=GRASSRGB color=bgyr
: [freeze]

I obviously don't understand the difference between the color table and the rgb_column. v.colors seems to lack the usual --overwrite.

So I have a workaround: remove the column by manual sql entry before running v.colors. The freeze is obviously a bug, and there seems to be a lack of clarity RE: the lack of --overwrite and the use of -r.

How you can reproduce this at home:

1. Open ​http://rlisdiscovery.oregonmetro.gov/

2. CLick on "Download" for 2000 Block Groups.

3. Save it into ~/tmp

4. unzip blockgrp2000.zip

5. grass70 -gui

6. Location Wizard

7. Project Location: rlus-bug-report

8. Read projection ... data file

9. Georef file: blockgrp2000.shp

10. Do *not* Import the data (although that is a nice grass 7 touch there!)

11. start grass.

12. db.connect driver=pg database="host=myhost,dbname=mydb"

[substitute your database of course]

13. v.in.ogr dsn=/home/powellj/tmp/blockgrp2000.shp output=blockgrp2000

(4385 primitives imported in about a minute).

14. v.colors --verbose map=blockgrp2000 column=pop08 layer=1 rgb_column=GRASSRGB color=bgyr

[works great, finishes in < 1s]

15. Repeat step 14. It freezes immediately, reading "no database is open".

[wenzeslaus]

Replying to jamesp670:

Hi Grass Devs,

You've got a big fan in me. I acknowledged GRASS on my AGU (see agu.org) poster this year and I'll make sure it's a cite next year.

That's great. I don't want to go off topic here in the ticket but please add it to

• ​http://grasswiki.osgeo.org/wiki/AGU_Fall_Meeting_2014:_GRASS_related_presentations_and_posters

[martinl]

On the first run v.colors creates column with lowercase (grassrgb) but on the second run the module checks given name (GRASSRGB) and tries to add column with lowercase again, and it fails. I changed v.colors in r63700 to be case-sensitive when using PG driver (similarly to SQLite driver).

Anyway I would highly suggest you not to use uppercase names for columns when using PG driver. Than you need to escape such columns, eg.

v.db.select soils_general where="\"GRASSRGB\" LIKE '255%'"

Commands like

v.db.select soils_general where="GRASSRGB LIKE '255%'"

will not work because the database driver will check grassrgb column (lowercase) instead of GRASSRGB.

[martinl]

No activity 20 months, closing, feel free to reopen if needed.