Opened 6 years ago

Closed 18 months ago

#2628 closed task (fixed)

Clarify use of PG variables used by loader_generate_script

Reported by: beland Owned by: komzpa
Priority: medium Milestone: PostGIS 2.5.0
Component: documentation Version: 2.1.x
Keywords: Cc:

Description

One of the steps in setting up the TIGER geocoder is to run, for example:

SELECT loader_generate_script(ARRAYMA?, 'sh') AS result;

The resulting script is buggy: it does not respect several of the configuration steps at the top, and some lines refer to non-existent files. I'm attaching a version with suggested fixes.

I am running postgis2_93-2.1.1-1.rhel6.x86_64 on CentOS 6.

Attachments (2)

ma_loader_improved.sh (3.4 KB) - added by beland 6 years ago.
ma_loader_improved_corrected.sh (13.6 KB) - added by beland 6 years ago.

Download all attachments as: .zip

Change History (14)

Changed 6 years ago by beland

Attachment: ma_loader_improved.sh added

comment:1 Changed 6 years ago by robe

Component: postgistiger geocoder
Owner: changed from pramsey to robe

comment:2 Changed 6 years ago by robe

What non-existent files does it refer to?

What configuration steps at the top does it not respect. It looks like you defined your own. e.g. I don't recall there being a variable called DOWNLOADDIR.

comment:3 Changed 6 years ago by beland

Actually, I think what I uploaded was derived from the wrong function; I'll fix that shortly.

The first item that is not respected properly is:

TMPDIR="/gisdata/temp/"

At the end of the first section there is:

cd /gisdata

The problem is that if you change TMPDIR, to say /home/beland/gisdata/temp, because /gisdata doesn't exist, the hard-coded references still point to /gisdata.

So what the script needs is something like $DOWNLOADDIR, and to define $TMPDIR=$DOWNLOADDIR/tmp. Or, change the meaning of TMPDIR to be what I call DOWNLOADDIR.

The second group of non-respected config variables are PGPORT, PGHOST, PGUSER, PGPASSWORD, and PGDATABASE. These are all defined but never used in the script.

I'm using local sockets, for example, so I don't actually want to have a PGHOST, so it made sense to me to just let the user define their own $PSQL command-line arguments. Or you could keep the PGXXXX variables and actually use them in the definition of $PSQL.

As for the non-existent files, there are often two "for" lines right next to each other, like:

for z in tl_*_25*_place.zip ; do $UNZIPTOOL -o -d $TMPDIR $z; done for z in */tl_*_25*_place.zip ; do $UNZIPTOOL -o -d $TMPDIR $z; done

The second pattern, */tl_*_25*_place.zip, does not match anything downloaded by the wget. There are */ variants in each of these pairs, and so far it seems they always produce an error. When cleaning up these scripts here, we just deleted the second lines and we still got the geocoding system to work, so I guess they aren't necessary.

Changed 6 years ago by beland

comment:4 Changed 6 years ago by beland

Oh, I see the PGXXXX variables are exported; maybe psql looks at them as environment variables? I suppose that works, though it is a bit confusing. Maybe worth a comment.

comment:5 Changed 6 years ago by robe

Yah I can link to the documentation on the PostgreSQL website for that. I'll add to readme.

For reference they are all defined here:

http://www.postgresql.org/docs/9.3/static/libpq-envars.html

comment:6 Changed 6 years ago by robe

Milestone: PostGIS 2.1.2PostGIS 2.2.0

comment:7 Changed 5 years ago by robe

Component: tiger geocoderdocumentation
Type: defecttask

changing this to a documentation task

comment:8 Changed 4 years ago by robe

Milestone: PostGIS 2.2.0PostGIS 2.1.9
Summary: loader_generate_script output is buggyClarify use of PG variables used by loader_generate_script

comment:9 Changed 2 years ago by pramsey

Milestone: PostGIS 2.1.9PostGIS 2.2.6

comment:10 Changed 2 years ago by robe

Milestone: PostGIS 2.2.6PostGIS 2.5.0

comment:11 Changed 18 months ago by komzpa

Owner: changed from robe to komzpa
Status: newassigned

comment:12 Changed 18 months ago by komzpa

Resolution: fixed
Status: assignedclosed

In 16652:

Clarify that PG* variables are used by psql directly.

Closes #2628

Note: See TracTickets for help on using tickets.