Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#1830 closed defect (wontfix)

pgsql2shp fix for "XInvalid endian flag value encountered"

Reported by: dfuhry2 Owned by: mcayland
Priority: medium Milestone: PostGIS 2.0.1
Component: loader/dumper Version: 2.0.x
Keywords: Cc:


pgsql2shp gives "Invalid endian flag value encountered." error on PostgreSQL 9.2 due to its expecting bytea_output to be in 'escape' format. At some point PostgreSQL changed the default bytea output format from 'escape' to 'hex'.

Attached patch sets server bytea_output to 'escape' while connecting, fixing the problem.

$ pgsql2shp ... Initializing... Done (postgis major version: 2). Output shape: Polygon Dumping: XInvalid endian flag value encountered.

Attachments (1)

invalid_endian_bytea_escape.v1.patch (586 bytes) - added by dfuhry2 7 years ago.

Download all attachments as: .zip

Change History (16)

Changed 7 years ago by dfuhry2

comment:1 Changed 7 years ago by pramsey

Resolution: fixed
Status: newclosed

Trunk at r9756, thanks!

comment:2 Changed 7 years ago by mcayland

Component: postgisloader/dumper
Priority: mediumhigh
Resolution: fixed
Status: closedreopened

NAK. This patch is broken and should be reverted because:

1) The default switch from 'escape' to 'hex' took place in PostgreSQL 9.0, so I'm not yet convinced that your diagnosis pointing at PostgreSQL 9.2 is correct

2) You've now broken dumping from older (< 9.0) PostgreSQL servers during a micro/patch revision

The symptoms suggest that your pgsql2shp client binary is linked against a pre-9.0 libpq somewhere else on your system, which is why libpq can't successfully interpret the 'hex' encoding being received from the server (we use libpq's internal PQescapeBytea() function rather than doing it ourselves).

In short, we need a better investigation of the problem and a revised patch to suit.

comment:3 Changed 7 years ago by robe

Yah I'll double-check but I don't recall having issues dumping and viewing my shape file in ArcGIS, QGIS, or OpenJump?. I'll have to double-check to make sure I didn't have an encoding setting on, but I don't think I did last I tried. Mark is right this bytea_output change happened in 9.0 not 9.2. So this could be possibly something else introduced in 9.2.

We should at the very least have in our make check routine a test that dumps out and then reloads the dumped shape file. I forget if any of our tests does that.

at anyrate, Paul you need to revert that patch since as Mark says -- it will break 8.4 support which we have to support in 2.0.

comment:4 Changed 7 years ago by pramsey

Ok, reverted at r9760

comment:5 Changed 7 years ago by dfuhry2

Linker information of my pgsql2shp executable:

$ ldd /usr/local/pgsql/bin/pgsql2shp =>  (0x00007fff5d4b2000) => /usr/local/lib/ (0x00007effe2e41000) => /lib/ (0x00007effe2ae0000) => /usr/lib/ (0x00007effe28b7000) => /usr/local/lib/ (0x00007effe2691000) => /usr/lib/ (0x00007effe244f000)
	/lib64/ (0x00007effe30a4000) => /usr/lib/ (0x00007effe21f9000) => /usr/lib/ (0x00007effe1e58000) => /usr/lib/ (0x00007effe1b90000) => /lib/ (0x00007effe198c000) => /usr/lib/ (0x00007effe1757000) => /lib/ (0x00007effe1520000) => /usr/lib/ (0x00007effe12d4000) => /lib/ (0x00007effe10b8000) => /usr/local/lib/ (0x00007effe0d1a000) => /usr/lib/ (0x00007effe0a05000) => /lib/ (0x00007effe0783000) => /lib/ (0x00007effe056d000) => /lib/ (0x00007effe0368000) => /usr/lib/ (0x00007effe0151000) => /usr/lib/ (0x00007effdff2b000) => /usr/lib/ (0x00007effdfd22000) => /lib/ (0x00007effdfb20000) => /lib/ (0x00007effdf90a000) => /usr/lib/ (0x00007effdf6fb000) => /usr/lib/ (0x00007effdf4e2000) => /usr/lib/ (0x00007effdf23f000) => /usr/lib/ (0x00007effdf02e000) => /usr/lib/ (0x00007effdedb6000) => /usr/lib/ (0x00007effdebb2000)
$ ls -l /usr/lib/
lrwxrwxrwx 1 root root 12 Jan 29 18:01 /usr/lib/ ->

comment:6 Changed 7 years ago by mcayland

Right so from my PostgreSQL 9.0 installation:

g90@kentang:~/rel/lib$ ls -l |grep libpq -rw-r--r-- 1 pg90 pg90 779062 Mar 26 12:38 libpq.a lrwxrwxrwx 1 pg90 pg90 12 Mar 26 12:38 -> lrwxrwxrwx 1 pg90 pg90 12 Mar 26 12:38 -> -rwxr-xr-x 1 pg90 pg90 440772 Mar 26 12:38

Given that your version of libpq is 5.2, I suspect that you're actually linking against a PostgreSQL 8.4 version of libpq instead which is why you see the error. Now the general philosophy of the PostgreSQL project is that newer versions of libraries can always talk to older versions of libraries, but not the other way around. My concern is that we can find ourselves going down a slippery and unsupported route by doing this...

Note that PostGIS 2.1 will no longer support PostgreSQL < 9.0 so this won't be a problem for newer versions moving forward; if you can still come up with a minimally invasive patch that uses version detection then I would consider applying it just to 2.0 branch.

comment:7 Changed 7 years ago by dfuhry2

Unfortunately we can't directly condition the change on PQlibVersion() since it's only available since 9.1: Also unfortunately, the PG_VERSION and PG_VERSION_NUM constants match the server version (9.2 in my case), not the libpq version. I guess PQlibVersion() could be tested for in configure, that still wouldn't distinguish between users of 9.0's libpq and 8.4's. Not clear to me how to make that distinction.

if (PQlibVersion() < 90000)

comment:8 Changed 7 years ago by mcayland

That does seem to be an excessively complicated approach - why not simply just always send "SET bytea_output = 'escape'" if PQserverVersion() returns something >= 9.0 (see

comment:9 Changed 7 years ago by mcayland

Actually on second thoughts, that will penalise people using a new version of libpq against a new PostgreSQL server since the point of hex encoding is that it noticeably reduces the traffic on the wire between client and server compared to escape encoding :(

Perhaps a better approach would be to perform PQunescapeBytea() on a hex-encoded string such as '\x4d' - if the client library supports hex encoding you'll get back an 'M', otherwise something different? I haven't tried this, but it might be worth an experiment.

comment:10 Changed 7 years ago by robe

Milestone: PostGIS 2.0.1PostGIS 2.0.2
Summary: pgsql2shp fix for "XInvalid endian flag value encountered" on PostgreSQL 9.2pgsql2shp fix for "XInvalid endian flag value encountered"

changing title -- it has nothing to do with 9.2 and is a libpq mismatch between server and client. I say push. All options sound not nice at this point and I question it affects that many people to begin with.

comment:11 Changed 7 years ago by robe

Owner: changed from pramsey to mcayland
Status: reopenednew

comment:12 Changed 7 years ago by robe

Priority: highmedium

comment:13 Changed 7 years ago by mcayland

Possibly the only sensible way to do this would be to have another command line option to force this setting where required, however because using older libpq is not a generally supported setup on PostgreSQL and we're not going to support 8.4 when 2.1 comes out anyway then I'm tempted to set this to 'wontfix'.

comment:14 Changed 7 years ago by robe

Resolution: wontfix
Status: newclosed

+1 - yes let's not fix.

I think the newer libpqs are already smart enough to work with older PostgreSQL. So it's only an issue for those trying to use an older libpq with a newer postgreSQL install. They shouldn't be doing that anyway and we shouldn't be encouraging it. Even PostgreSQL for example doesn't allow you to backup/restore a PostgreSQL database with an older pg_dump/pg_restore. I see this as much the same thing :)

comment:15 Changed 7 years ago by robe

Milestone: PostGIS 2.0.2PostGIS 2.0.1
Note: See TracTickets for help on using tickets.