Opened 6 years ago

Closed 4 years ago

#2336 closed defect (fixed)

FIPS 20 causes wildcard expansion to wget all files

Reported by: skipl Owned by: robe
Priority: high Milestone: PostGIS 2.1.9
Component: tiger geocoder Version: trunk
Keywords: Cc: jhughesjr

Description

Postgis 2.1.0 beta 2, tiger_geocoder extension, tiger2012 dataset:

In loader_generate_script() the pattern for generating the per-state wget line is '/*_'
s.state_fips. When processing Kansas (FIPS 20), the expansion matches '20' in the year 2012.
This results in downloading the entire nation from each directory (i.e. *_20 matches tl_2012*). Tightening the pattern to '/tl_*_'
s.state_fips does not appear to cause any problems with the rest of the census tiger2012 directory structure.

PostgreSQL 9.2.4 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu/Linaro? 4.6.3-1ubuntu5) 4.6.3, 64-bit POSTGIS="2.1.0beta2 r11441" GEOS="3.3.8-CAPI-1.7.8" PROJ="Rel. 4.8.0, 6 March 2012" GDAL="GDAL 1.10.0, released 2013/04/24" LIBXML="2.7.8" RASTER

Change History (11)

comment:1 Changed 6 years ago by skipl

Version: 2.0.xtrunk

comment:2 Changed 6 years ago by robe

Priority: mediumblocker

skipl thanks for the info. I plan to get this fixed before the next beta comes out.

comment:3 Changed 6 years ago by robe

Resolution: fixed
Status: newclosed

fixed at r11548

comment:4 Changed 5 years ago by jhughesjr

Cc: jhughesjr added
Priority: blockerhigh
Resolution: fixed
Status: closedreopened

This issue appears to have only been partially fixed in version: POSTGIS 2.1.3. When loading KS - the following wget commands are still loading all files and not using the tighter '/tl_*_' pattern:

wget ftp://ftp2.census.gov/geo/tiger/TIGER2010/ZCTA5/2010/*_20* --no-parent --relative --recursive --level=2 --accept=zip --mirror --reject=html wget ftp://ftp2.census.gov/geo/tiger/TIGER2013/FACES/*_20* --no-parent --relative --recursive --level=2 --accept=zip --mirror --reject=html wget ftp://ftp2.census.gov/geo/tiger/TIGER2013/FEATNAMES/*_20* --no-parent --relative --recursive --level=2 --accept=zip --mirror --reject=html wget ftp://ftp2.census.gov/geo/tiger/TIGER2013/EDGES/*_20* --no-parent --relative --recursive --level=2 --accept=zip --mirror --reject=html wget ftp://ftp2.census.gov/geo/tiger/TIGER2013/ADDR/*_20* --no-parent --relative --recursive --level=2 --accept=zip --mirror --reject=html

comment:5 Changed 4 years ago by robe

Milestone: PostGIS 2.1.0PostGIS 2.1.5

Thanks for reporting. Will try to fix in next couple of days. I was debating of just upgrading upcoming 2.1.5 to tiger 2014 which I have already in PostGIS 2.2 since 2.2 is still a ways away. I imagine the same issue exists there too.

comment:6 Changed 4 years ago by pramsey

Milestone: PostGIS 2.1.5PostGIS 2.1.6

comment:7 Changed 4 years ago by pramsey

Milestone: PostGIS 2.1.6PostGIS 2.1.7

comment:8 Changed 4 years ago by robe

Milestone: PostGIS 2.1.7PostGIS 2.1.8

comment:9 Changed 4 years ago by robe

Milestone: PostGIS 2.1.8PostGIS 2.2.0

I have to check if this is still an issue

comment:10 Changed 4 years ago by robe

Milestone: PostGIS 2.2.0PostGIS 2.1.9

comment:11 Changed 4 years ago by robe

Resolution: fixed
Status: reopenedclosed

Fixed for postgis 2.1.9 at r13989 and PostGIS 2.2 at r13990

Note: See TracTickets for help on using tickets.