Opened 7 years ago

Closed 7 years ago

#294 closed defect (fixed)

Recoding non-Latin characters with GDAL is not supported now.

Reported by: akaginch Owned by: warmerdam
Priority: major Component: Package
Version: Keywords: character recoding
Cc:

Description

The encoding of Shapefile has been supported in GDAL since 1.9, but it is not enabled because iconv library is not linked yet. Therefore, non-Latin language users sometimes have troubles.

For example, when I convert a dataset of Shapefile that dbf file has "19" value (it means "CP932") in LDID field to KML format with ogr2ogr, the following message is shown.

Warning1: Recode from CP932 to UTF-8 not supported, treated as ISO8859-1 to UTF-8

The Japanese characters of generated KML file is incorrect because of wrong recoding that this message says. I think this also results character corruption in QGIS as below.

Quantum GIS Desktop - Bug #5255: Wrong codepage of shapefile - QGIS Issue Tracking

Would you please configure the packages including GDAL with iconv library and make the benefit by version-up come off?

Best regards.

Change History (8)

comment:2 Changed 7 years ago by akaginch

My built gdal19.dll for testing is at https://dl.dropbox.com/u/21526091/gdal/binaries.html. You can replace OSGeo4W/bin/gdal19.dll with the downloaded gdal19.dll and test GDAL 1.9.1 with iconv library linked now.

comment:3 Changed 7 years ago by akaginch

I noticed that this problem had been already solved in http://trac.osgeo.org/gdal/ticket/4650. I hope that the gdal package will be updated as soon as possible.

comment:4 Changed 7 years ago by akaginch

This is a supplementation of previous post. Changes in GDAL #4650 including r24514 "Windows builds of GDAL can support some recoding from/to CPxxx even without iconv" works well. No problem of recoding CP932 to UTF-8. The iconv library would not necessary with latest trunk.
Best regards.

comment:5 Changed 7 years ago by rouault

I think that the osgeo4w effort should concentrate on GDAL 1.9.x for now, and "just" rebuild and link against iconv, which will solve most problems.

The trunk capability of using windows API in some cases is cool, but realistically GDAL 2.0 will not be released before a few months, or maybe one year.

comment:6 Changed 7 years ago by warmerdam

Owner: changed from osgeo4w-dev@… to warmerdam

I'll take a crack at this.

comment:7 Changed 7 years ago by warmerdam

Status: newassigned

The OSGeo4W gdal-1.9.1-2 package is now built against iconv. I would appreciate it if someone could test things and close the ticket if the issue is resolved.

comment:8 Changed 7 years ago by akaginch

Resolution: fixed
Status: assignedclosed

Thank you warmerdam.

I've tested new gdal19.dll with ogr2ogr as below. The encoding of source shapefile is CP932 and the LDID is set to 19.
ogr2ogr -f KML test.kml test.shp
It has converted character encoding from CP932 to UTF-8 well.

Note: See TracTickets for help on using tickets.