Opened 8 years ago

Closed 8 years ago

Last modified 8 years ago

#3660 closed defect (invalid)

Cannot import shapefile due to encoding problem

Reported by: miguelluaces Owned by: strk
Priority: blocker Milestone: PostGIS 2.3.1
Component: liblwgeom Version: 2.3.x
Keywords: shp2pgsql Cc:

Description

A shapefile that could be imported without problems in shp2pgsql 2.1.7 cannot be imported in shp2pgsql 2.3.0. The execution fails with the following error:

Unable to convert data value to UTF-8 (iconv reports "Illegal byte sequence"). Current encoding is "WINDOWS-1252". Try "LATIN1" (Western European), or one of the values described at http://www.postgresql.org/docs/current/static/multibyte.html.

The commandline was:

d:\PostgreSQL\9.5\bin\shp2pgsql.exe -W UTF-8 establecimiento.shp

Attachments (1)

establecimiento.zip (116.8 KB ) - added by miguelluaces 8 years ago.
Shapefile extracted from OpenStreetMap that worked without problems in shp2pgsql 2.1.7

Download all attachments as: .zip

Change History (3)

by miguelluaces, 8 years ago

Attachment: establecimiento.zip added

Shapefile extracted from OpenStreetMap that worked without problems in shp2pgsql 2.1.7

comment:1 by robe, 8 years ago

Resolution: invalid
Status: newclosed

As the note says, try LATIN1.

This works fine for me with:

shp2pgsql.exe -W LATIN1 establecimiento.shp 

I can't see why UTF8 would ever work with this file in 2.1.7

comment:2 by miguelluaces, 8 years ago

I should have written explicitly that the dataset is UTF-8, not LATIN1. If you open the dbf file in LibreOffice using Latin1 as encoding you can see that row number 3 is shown as

Muíño Daniel o Ferreiro

instead of :

Muíño Daniel o Ferreiro

What should I do now? Compile it myself and set a breakpoint to see which is the offending character? I would rather have the error messages improved to show me which is the offending row so that I can try to fix it.

And again, the same shapefile can be imported with shp2pgsql 2.1.7 with the same commandline without errors.

Note: See TracTickets for help on using tickets.