Opened 18 years ago
Last modified 18 years ago
#1762 new defect
Running mapserver code at different locale may result in unexpected behaviour
Reported by: | szekerest | Owned by: | sdlime |
---|---|---|---|
Priority: | high | Milestone: | |
Component: | MapServer C Library | Version: | 4.8 |
Severity: | major | Keywords: | |
Cc: |
Description
I had a problem with that mapserver/mapscript is working under the locale context of the host process, but the code is not ready to operate in all locales so far. For example the lexer expects dot as the decimal separator and some of the locales use comma and atof will truncate the numbers to the integer part causing mystical errors. As a quick fix I suggest to call setlocale to the default "C" at the msSetup initialization function. The best solution however would rewriting the code to be locale independent if it is expected. Fixing this bug may also fix http://mapserver.gis.umn.edu/bugs/show_bug.cgi?id=1758 Tamas
Attachments (1)
Change History (12)
comment:1 by , 18 years ago
Cc: | added |
---|
comment:3 by , 18 years ago
Cc: | added |
---|
comment:4 by , 18 years ago
From man 7 locale: If the second argument to setlocale() is empty string, "", for the default locale, it is determined using the following steps: 1. If there is a non-null environment variable LC_ALL, the value of LC_ALL is used. 2. If an environment variable with the same name as one of the categories above exists and is non- null, its value is used for that category. 3. If there is a non-null environment variable LANG, the value of LANG is used. I am attaching a simple c program that proves that atof can be fully localized so that it should not be necessary to hard code the locale to C.
by , 18 years ago
Sample program to play around with atof locale support
comment:5 by , 18 years ago
Umberto, I'm not sure what the demonstration program is intended to demonstrate. I would add that atof() calls in MapServer itself are not the only problem. We also have issues with libraries called by MapServer.
comment:6 by , 18 years ago
Frank, the bug report says: > For example > the lexer expects dot as the decimal separator and some of > the locales use comma and atof will truncate the numbers to > the integer part causing mystical errors. and I only intended to show that atof when properly invoked can deal all locales that it knows of. The setlocale(LC_NUMERIC,"") call will of course affect all other libraries as would the setlocale(LC_NUMERIC,"C").
comment:7 by , 18 years ago
Cc: | added |
---|
comment:8 by , 18 years ago
Umberto, I think we have the inverse problem now. Because the lexer expects dot as the decimal separator atof should be "delocalized" so as to parse the numeric string expecting dot regardless of the current locale setting. Furthermore the code should run properly even if the host process or some other libraries are tampering the locale setting of the process. (By the way how could we set the locale of the java runtime without modifying the java code? "-Duser.language=.. -Duser.region=.." in the command line does not seem to work) Tamas
comment:9 by , 18 years ago
> I think we have the inverse problem now. Because the lexer expects dot as the > decimal separator atof should be "delocalized" so as to parse the numeric string > expecting dot regardless of the current locale setting. I understand, I have looked at the source. > (By the way how could we set the locale of the java runtime without modifying > the java code? "-Duser.language=.. -Duser.region=.." in the command line does > not seem to work) Set LANG and LC_* variables to the locale you want (for example en_US or C to allow . as decimal separator). Please note that Java mapscript will work even under locales like it_IT that define , as decimal separator.
comment:10 by , 18 years ago
>Set LANG and LC_* variables to the locale you want (for example en_US or C to >allow . as decimal separator). Please note that Java mapscript will work even >under locales like it_IT that define , as decimal separator. I could not get it to work. The java runtime is using the C locale regardless of the LANG and LC_* exports ;) To make sure of it i have printed the return value of "setlocale(LC_NUMERIC, NULL)" somewhere in the mapscript C code. I shall continue to believe that make test for java would also fail if we can set the locale context of java to a setting having the decimal separator other than "." Tamas
comment:11 by , 18 years ago
> I could not get it to work. The java runtime is using the C locale regardless of > the LANG and LC_* exports ;) To make sure of it i have printed the return value > of "setlocale(LC_NUMERIC, NULL)" somewhere in the mapscript C code. Tamas, when I say that Java obeys to LANG and C_* settings I mean the jvm and PURE java classes (like java.lang.String, Double, etc), not mapscript. Java mapscript is written in C and Java does not propagate those settings to it. The only way Java mapscript can be made aware of LANG and/or LC_* is by calling setlocale(LC_ALL,""); in mapscript itself. So Java mapscript does not suffer of this problem because it is using the system locale (in my case it_IT) while doing its work in the jvm, but will let mapscript work in C mode (which should be the default until setlocale is called, which is never done in mapscript AFAIK). This probably happens because Java never calls setlocale in its own C code. On the other hand I suppose that the Mono interpreter is calling setlocale somewhere and thus influencing the behaviour of C# mapscript. > I shall continue to believe that make test for java would also fail if we can > set the locale context of java to a setting having the decimal separator other > than "." You are right. If, for instance, I put a call to setlocale(LC_ALL,"") in maputil.c Java mapscript exhibits the same behaviour of C# mapscript which makes it impossibile to load a file that has ',' as the decimal separator. I hope I have answered your question. Umberto
Note:
See TracTickets
for help on using tickets.