Opened 11 years ago

Closed 8 years ago

#4135 closed defect (fixed)

CPLRecodeFromWCharIconv() does not work properly if wchar_t is 4bytes

Reported by: warmerdam Owned by: warmerdam
Priority: normal Milestone:
Component: default Version: svn-trunk
Severity: normal Keywords: recode
Cc: dron

Description

If a source encoding of CPL_ENC_UCS2 (or CPL_ENC_UTF16) is used with CPLRecodeFromWChar() and the call is passed to the iconv implementation then it will not work properly.

iconv() itself deduces the character size from the source encoding, but when working with wchar_t the character size should be determined based on the size of wchar_t instead of the encoding. That is, it should be ok to use CPL_ENC_UTF16 even when the characters are held in 32bit wchar_t's.

The solution, I believe, is to translate the wchar_t's into a 16bit buffer within the iconv implementation. It would be ideal if there was a way of asking iconv the character size for a particular encoding.

For that matter, it should be acceptable to use CPL_ENC_UTF8 in wchar_t buffers!

Note that the iconv() support was not present in 1.8 so this is a trunk-only issue.

Change History (5)

comment:1 by warmerdam, 11 years ago

Status: newassigned

I'll take a crack at resolving this - advice welcome.

comment:2 by dron, 11 years ago

Frank,

I've just started looked at the issue and found your implementation. I think it should be a correct workaround for this issue if we are sure that wide character string always comes from the same system eliminating the possible endiannes problem.

comment:3 by Even Rouault, 11 years ago

For the record: r22600 - preliminary support for wchar_t with iconv recode (#4135)

comment:4 by Jukka Rahkonen, 8 years ago

It looks like r22600 has been there in the code for several years now. Dron considered it as a correct workaround. Would it be OK to close this issue as fixed?

comment:5 by Jukka Rahkonen, 8 years ago

Resolution: fixed
Status: assignedclosed

Closinh now.

Note: See TracTickets for help on using tickets.