Opened 6 years ago

Closed 3 years ago

#4135 closed defect (fixed)

CPLRecodeFromWCharIconv() does not work properly if wchar_t is 4bytes

Reported by: warmerdam Owned by: warmerdam
Priority: normal Milestone:
Component: default Version: svn-trunk
Severity: normal Keywords: recode
Cc: dron

Description

If a source encoding of CPL_ENC_UCS2 (or CPL_ENC_UTF16) is used with CPLRecodeFromWChar() and the call is passed to the iconv implementation then it will not work properly.

iconv() itself deduces the character size from the source encoding, but when working with wchar_t the character size should be determined based on the size of wchar_t instead of the encoding. That is, it should be ok to use CPL_ENC_UTF16 even when the characters are held in 32bit wchar_t's.

The solution, I believe, is to translate the wchar_t's into a 16bit buffer within the iconv implementation. It would be ideal if there was a way of asking iconv the character size for a particular encoding.

For that matter, it should be acceptable to use CPL_ENC_UTF8 in wchar_t buffers!

Note that the iconv() support was not present in 1.8 so this is a trunk-only issue.

Change History (5)

comment:1 Changed 6 years ago by warmerdam

Status: newassigned

I'll take a crack at resolving this - advice welcome.

comment:2 Changed 6 years ago by dron

Frank,

I've just started looked at the issue and found your implementation. I think it should be a correct workaround for this issue if we are sure that wide character string always comes from the same system eliminating the possible endiannes problem.

comment:3 Changed 6 years ago by Even Rouault

For the record: r22600 - preliminary support for wchar_t with iconv recode (#4135)

comment:4 Changed 3 years ago by Jukka Rahkonen

It looks like r22600 has been there in the code for several years now. Dron considered it as a correct workaround. Would it be OK to close this issue as fixed?

comment:5 Changed 3 years ago by Jukka Rahkonen

Resolution: fixed
Status: assignedclosed

Closinh now.

Note: See TracTickets for help on using tickets.