Opened 10 years ago
Closed 9 years ago
#3060 closed defect (fixed)
loader_load_staged_data relying on undefined behaviour when generating column lists
Reported by: | gibreel | Owned by: | robe |
---|---|---|---|
Priority: | medium | Milestone: | PostGIS 2.3.0 |
Component: | tiger geocoder | Version: | 2.1.x |
Keywords: | loader | Cc: |
Description
The loader_load_staged_data function assumes that select statement for column names will produce the same sequence for both the source and destination tables. While it might, such ordering behaviour isn't guaranteed as the error below shows.
The simple fix is to add 'order by column_name' to both queries. Patch attached.
Trace output below:
+ psql -c 'ALTER TABLE tiger_staging.county RENAME geoid TO cntyidfp; SELECT loader_load_staged_data(lower('\''county'\''), lower('\''county_all'\''));' NOTICE: INSERT INTO tiger_data.county_all(statefp,countyfp,countyns,cntyidfp,name,namelsad,lsad,classfp,mtfcc,csafp,cbsafp,metdivfp,funcstat,aland,awater,intptlat,intptlon,the_geom) SELECT statefp,countyfp,countyns,name,namelsad,lsad,classfp,mtfcc,csafp,cbsafp,metdivfp,funcstat,aland,awater,intptlat,intptlon,the_geom,cntyidfp FROM tiger_staging.county; CONTEXT: SQL function "loader_load_staged_data" statement 1 ERROR: column "awater" is of type double precision but expression is of type character varying LINE 1: ...mtfcc,csafp,cbsafp,metdivfp,funcstat,aland,awater,intptlat,i... ^ HINT: You will need to rewrite or cast the expression. QUERY: INSERT INTO tiger_data.county_all(statefp,countyfp,countyns,cntyidfp,name,namelsad,lsad,classfp,mtfcc,csafp,cbsafp,metdivfp,funcstat,aland,awater,intptlat,intptlon,the_geom) SELECT statefp,countyfp,countyns,name,namelsad,lsad,classfp,mtfcc,csafp,cbsafp,metdivfp,funcstat,aland,awater,intptlat,intptlon,the_geom,cntyidfp FROM tiger_staging.county; CONTEXT: PL/pgSQL function loader_load_staged_data(text,text,text[]) line 24 at EXECUTE statement
Attachments (1)
Change History (6)
by , 10 years ago
Attachment: | loader_load_data.udiff added |
---|
comment:1 by , 10 years ago
Milestone: | → PostGIS 2.1.6 |
---|
gibreel,
Thanks. Probably won't have time to look at it for another week.
comment:2 by , 10 years ago
Milestone: | PostGIS 2.1.6 → PostGIS 2.1.7 |
---|
comment:3 by , 10 years ago
Milestone: | PostGIS 2.1.7 → PostGIS 2.1.8 |
---|
comment:4 by , 9 years ago
Milestone: | PostGIS 2.1.8 → PostGIS 2.3.0 |
---|
I'm not sure if I want to change this and haven't run into this issue.
comment:5 by , 9 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
I've fixed this already since I ran into the same issue logged in #3347 when I had a lot of tables. I ended up ordering by ordinal_position instead of column name at r14317 (and other branches)
`
ORDER BY ordinal_position
`
The reason is that the names might actually change but sequence of order is more important.
Patch for tiger.loader_load_data to fix column order mismatch errors