Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#908 closed defect (fixed)

Unix/Linux sh bash version of tiger2010 geocoder loader does not work

Reported by: donh Owned by: robe
Priority: medium Milestone: PostGIS 2.0.0
Component: tiger geocoder Version: master
Keywords: history Cc:

Description

ctrl-m's in the file and/or generated by psql would crash psql. Missing command line parameters would also crash it. I would generate my load_tiger script by running psql as: \a \o load_tiger \i states \o \q I would them remove the control-m's from load_tiger. tr "\r" "\n" < load_tiger > load_tiger2 I then would edit load_tiger2 putting a #!bin/bash at the top and removing the line numbers at the end. I had already downloaded the tiger database files so I would replace wget with #wget in the file. shp2pgsql does not really convert latin1 to utf-8 all the time so I added a "| iconv -f latin1 -t UTF8 | $PGBIN/psql" after the shp2pgsql command. That did away with the aborted transactions from postgresql.

Attachments (5)

tiger2010_diff (40.5 KB) - added by donh 10 years ago.
diff to fix the problem
load_tiger4 (585.0 KB) - added by donh 10 years ago.
generated file that I used to load the tiger database
states (309 bytes) - added by donh 10 years ago.
file that lists states to process; some territories are missing
tiger_loader.sql2 (20.9 KB) - added by donh 10 years ago.
my tiger_loader.sql file
reverse_geocode.sql (5.1 KB) - added by donh 10 years ago.
geocode/reverse_geocode.sql original had embedded character code which crashed the program

Download all attachments as: .zip

Change History (11)

Changed 10 years ago by donh

Attachment: tiger2010_diff added

diff to fix the problem

Changed 10 years ago by donh

Attachment: load_tiger4 added

generated file that I used to load the tiger database

Changed 10 years ago by donh

Attachment: states added

file that lists states to process; some territories are missing

comment:1 Changed 10 years ago by mcayland

shp2pgsql does not really convert latin1 to utf-8 all the time so I added a "| iconv -f latin1 -t UTF8 | $PGBIN/psql" after the shp2pgsql command. That did away with the aborted transactions from postgresql.

If this is true, then you've found a bug. Can it be reproduced using the files attached to this ticket?

Changed 10 years ago by donh

Attachment: tiger_loader.sql2 added

my tiger_loader.sql file

Changed 10 years ago by donh

Attachment: reverse_geocode.sql added

geocode/reverse_geocode.sql original had embedded character code which crashed the program

comment:2 Changed 10 years ago by robe

Version: 1.5.Xtrunk

comment:3 Changed 10 years ago by robe

Mark,

What he discovered is a known and closed bug #808. I confirmed already he is running a version of shp2pgsql before the fix (he's running a version from December 2010 and the fixed version should be dated after March 18, 2011

Refer to: http://www.postgis.org/pipermail/postgis-users/2011-April/029406.html

comment:4 Changed 10 years ago by robe

Status: newassigned
Summary: linux version of tiger2010 geocoder does not workUnix/Linux sh bash version of tiger2010 geocoder loader does not work

Most if not all these issues have been fixed peacemeal. I'm hoping last fix at r7278 will cover what commpatibility issues are left.

Will know once done testing on my CentOS box.

comment:5 Changed 10 years ago by robe

Resolution: fixed
Status: assignedclosed

I was successful in loading California data on a CentOS with the revised changes in r7278 so consider this a done deal.

comment:6 Changed 10 years ago by robe

Keywords: history added
Note: See TracTickets for help on using tickets.