Opened 10 years ago

Closed 9 years ago

#3060 closed defect (fixed)

loader_load_staged_data relying on undefined behaviour when generating column lists

Reported by: gibreel Owned by: robe
Priority: medium Milestone: PostGIS 2.3.0
Component: tiger geocoder Version: 2.1.x
Keywords: loader Cc:

Description

The loader_load_staged_data function assumes that select statement for column names will produce the same sequence for both the source and destination tables. While it might, such ordering behaviour isn't guaranteed as the error below shows.

The simple fix is to add 'order by column_name' to both queries. Patch attached.

Trace output below:

+ psql -c 'ALTER TABLE tiger_staging.county RENAME geoid TO cntyidfp;  SELECT loader_load_staged_data(lower('\''county'\''), lower('\''county_all'\''));'
NOTICE:  INSERT INTO tiger_data.county_all(statefp,countyfp,countyns,cntyidfp,name,namelsad,lsad,classfp,mtfcc,csafp,cbsafp,metdivfp,funcstat,aland,awater,intptlat,intptlon,the_geom) SELECT statefp,countyfp,countyns,name,namelsad,lsad,classfp,mtfcc,csafp,cbsafp,metdivfp,funcstat,aland,awater,intptlat,intptlon,the_geom,cntyidfp FROM tiger_staging.county;
CONTEXT:  SQL function "loader_load_staged_data" statement 1
ERROR:  column "awater" is of type double precision but expression is of type character varying
LINE 1: ...mtfcc,csafp,cbsafp,metdivfp,funcstat,aland,awater,intptlat,i...
                                                             ^
HINT:  You will need to rewrite or cast the expression.
QUERY:  INSERT INTO tiger_data.county_all(statefp,countyfp,countyns,cntyidfp,name,namelsad,lsad,classfp,mtfcc,csafp,cbsafp,metdivfp,funcstat,aland,awater,intptlat,intptlon,the_geom) SELECT statefp,countyfp,countyns,name,namelsad,lsad,classfp,mtfcc,csafp,cbsafp,metdivfp,funcstat,aland,awater,intptlat,intptlon,the_geom,cntyidfp FROM tiger_staging.county;
CONTEXT:  PL/pgSQL function loader_load_staged_data(text,text,text[]) line 24 at EXECUTE statement

Attachments (1)

loader_load_data.udiff (926 bytes ) - added by gibreel 10 years ago.
Patch for tiger.loader_load_data to fix column order mismatch errors

Download all attachments as: .zip

Change History (6)

by gibreel, 10 years ago

Attachment: loader_load_data.udiff added

Patch for tiger.loader_load_data to fix column order mismatch errors

comment:1 by robe, 10 years ago

Milestone: PostGIS 2.1.6

gibreel,

Thanks. Probably won't have time to look at it for another week.

comment:2 by pramsey, 10 years ago

Milestone: PostGIS 2.1.6PostGIS 2.1.7

comment:3 by robe, 10 years ago

Milestone: PostGIS 2.1.7PostGIS 2.1.8

comment:4 by robe, 9 years ago

Milestone: PostGIS 2.1.8PostGIS 2.3.0

I'm not sure if I want to change this and haven't run into this issue.

comment:5 by robe, 9 years ago

Resolution: fixed
Status: newclosed

I've fixed this already since I ran into the same issue logged in #3347 when I had a lot of tables. I ended up ordering by ordinal_position instead of column name at r14317 (and other branches)

`

ORDER BY ordinal_position

`

The reason is that the names might actually change but sequence of order is more important.

Note: See TracTickets for help on using tickets.