Problem with Curved labels and multibyte characters

[vulukut@…]

Curved labels don't take multi-byte and/or special characters into account and 
does not show the right characters for i.e "þ,ç,ý,ð" but shows characters 
like "Ã,§,Ö,±"
here is the related map file part :
   LABEL
        TYPE TRUETYPE
        FONT "fritqat"
        SIZE 7
        MINSIZE 4
        MAXSIZE 256
        POSITION cc
        OFFSET 0 0 
        ANGLE FOLLOW
        BUFFER 10
        MINDISTANCE -1
        MINFEATURESIZE -1
        COLOR 0 0 0
        OUTLINECOLOR 254 254 254
        ANTIALIAS TRUE
        PARTIALS FALSE
        FORCE FALSE
	ENCODING "ISO-8859-9"
      END

the related row of the font file:
fritqat                         Vera.ttf

on a side note, these characters show normal with labels which doesn't follow 
lines.

[vulukut@…]

example screenshot of the bug

[dmorissette]

Too late for 4.10.0. Setting target milestone to 5.0.

Depending on the amount of changes involved with the fix we may or may not
backport to 4.10.x

[craign@…]

I've experienced the same issue on MS4W 2.2.2 (MapServer 4.10.0) as well as 
MS4W 2.2.3 (MapServer 4.10.1). I believe it is the same issue as my Unicode/
UTF8 stored characters (VISCII and BIG-HKSCS) are being corrupted whenever I 
set the ANGLE to FOLLOW but it works fine on AUTO.

Is there any chance that this bug could be fixed earlier than version 5.0?

[sdlime]

Benj: Do you have any thoughts on where to start with this? I can persue the 
fix but could use a jump start.

Steve

[dmorissette]

I didn't look deep into this, but I suspect the curved label code will need to
be modified to use iconv() to read characters from the string instead of just
assuming that each byte is a separate char:

http://www.gnu.org/software/libiconv/documentation/libiconv/iconv.3.html

A good start would be to look at how iconv() is used in msGetEncodedString()
(mapgd.c). It might be as simple as borrowing some of that code.

[sdlime]

Dan: Could we borrow the entire function to convert the string to it's encoded
form before processing by the curved label functions? I'm not familiar with
iconv myself. If I understand this, and I probably don't, then I'm kinda
surprised the regular processing, especially placement (e.g. POSITION CC),
works. Without passing the encoded string to the gd/freetype size computation
functions I can't see how we're getting the right sizes for placement. 

Could the character set conversion be done before a label is added to the cache?

Also, anyone got a good test case?

Steve

[craign@…]

I'm sure I could supply you with a layer and map file demonstrating the 
problem. Should I e-mail it to you?

[sdlime]

Actually unless it's huge you can attach it here. I can only handle 5Mb
attachments at work. Alternatively you can ftp it to
ftp.dnr.state.mn.us/pub/incoming and let me know where you drop it.

Steve

[sdlime]

Folks: I'm wanting to work on this but have no sample data to test with. I need:

• ttf font
• line data with multibyte chars
• sample mapfile

Can anyone provide that?

Steve

[novorado]

Replying to sdlime:

Hi Steve,

how I can contact you by email to provide relevant data with cyrillic characters? I believe fix should be easy, it is really loosing encoding when follow option uses algorithm to make lable non-linear (curved)

Best regards, Dmitry

Folks: I'm wanting to work on this but have no sample data to test with. I need:

• ttf font
• line data with multibyte chars
• sample mapfile

Can anyone provide that?

Steve

[novorado]

1921 screenshot with Cyrillic data

[sdlime]

Getting there. With older code if you draw text with the extended character sets you'll notice that placement is off because the iconv isn't being run before placement is computed. I committed a fix for that piece so now POSITION is correct. Paths don't work yet though. Perhaps the storage model we're using isn't sufficient...

Steve

[dmorissette]

An update on this bug: Steve has committed some changes to deal with multibyte chars in the ANGLE FOLLOW code which was working but relied on setlocale. We then added in r6242 two new functions msGetNextUTF8Char() and msGetNumUTF8Chars() in mapstring.c which are independent of setlocale and are used in mapprimitive.c and mapgd.c.

This is mostly fixed in SVN trunk now, just need more testing.

[dmorissette]

Steve, have you had a chance to test with my modifs (SVN trunk)? Can this bug be closed?

[sdlime]

Works with my simple test case. Would be nice to get confirmation from other folks. Closing for now...

Steve