encoding information in locale gets lost

[mlennert]

My locale settings before launching GRASS 7:

$ locale
LANG=fr_BE.UTF-8
LANGUAGE=fr_BE:fr
LC_CTYPE="fr_BE.UTF-8"
LC_NUMERIC="fr_BE.UTF-8"
LC_TIME="fr_BE.UTF-8"
LC_COLLATE="fr_BE.UTF-8"
LC_MONETARY="fr_BE.UTF-8"
LC_MESSAGES="fr_BE.UTF-8"
LC_PAPER="fr_BE.UTF-8"
LC_NAME="fr_BE.UTF-8"
LC_ADDRESS="fr_BE.UTF-8"
LC_TELEPHONE="fr_BE.UTF-8"
LC_MEASUREMENT="fr_BE.UTF-8"
LC_IDENTIFICATION="fr_BE.UTF-8"
LC_ALL=

locale settings after launching GRASS 7:

> locale
LANG=fr_BE
LANGUAGE=fr_BE
LC_CTYPE=fr_BE
LC_NUMERIC=C
LC_TIME=fr_BE
LC_COLLATE=fr_BE
LC_MONETARY=fr_BE
LC_MESSAGES=fr_BE
LC_PAPER=fr_BE
LC_NAME=fr_BE
LC_ADDRESS=fr_BE
LC_TELEPHONE=fr_BE
LC_MEASUREMENT=fr_BE
LC_IDENTIFICATION=fr_BE
LC_ALL=

Thus the special characters in translated messages are not displayed correctly.

This is with svn revision 56918.

Moritz

[mlennert]

Replying to mlennert:

My locale settings before launching GRASS 7:

$ locale
LANG=fr_BE.UTF-8
LANGUAGE=fr_BE:fr
LC_CTYPE="fr_BE.UTF-8"
LC_NUMERIC="fr_BE.UTF-8"
LC_TIME="fr_BE.UTF-8"
LC_COLLATE="fr_BE.UTF-8"
LC_MONETARY="fr_BE.UTF-8"
LC_MESSAGES="fr_BE.UTF-8"
LC_PAPER="fr_BE.UTF-8"
LC_NAME="fr_BE.UTF-8"
LC_ADDRESS="fr_BE.UTF-8"
LC_TELEPHONE="fr_BE.UTF-8"
LC_MEASUREMENT="fr_BE.UTF-8"
LC_IDENTIFICATION="fr_BE.UTF-8"
LC_ALL=

locale settings after launching GRASS 7:

> locale
LANG=fr_BE
LANGUAGE=fr_BE
LC_CTYPE=fr_BE
LC_NUMERIC=C
LC_TIME=fr_BE
LC_COLLATE=fr_BE
LC_MONETARY=fr_BE
LC_MESSAGES=fr_BE
LC_PAPER=fr_BE
LC_NAME=fr_BE
LC_ADDRESS=fr_BE
LC_TELEPHONE=fr_BE
LC_MEASUREMENT=fr_BE
LC_IDENTIFICATION=fr_BE
LC_ALL=

Thus the special characters in translated messages are not displayed correctly.

This is with svn revision 56918.

The culprit seems to be r52517 which includes the following line in lib/init.grass.py:

language = language.split('.')[0] # Split off ignored .encoding part if present

Why should the encoding part be split off, especially since it is _not_ ignored as implied by the comment. Module messages that have been translated (in my case into French) and include special characters are not displayed correctly.

When I comment out this line, I get messages in the correct encoding.

Maris, could you explain the reason for taking away the encoding info ? I'm tempted to comment out the line for now and see if anyone reports negative consequences, but I'll wait a few days to see if there is a reason for it.

[mlennert]

This seems to have been fixed by r59163, so closing.

[mlennert]

Reopening this, as with a fairly fresh trunk (r65364), I get:

Before running the grass startup script:

$ locale
LANG=fr_BE.utf8
LANGUAGE=
LC_CTYPE="fr_BE.utf8"
LC_NUMERIC="fr_BE.utf8"
LC_TIME="fr_BE.utf8"
LC_COLLATE="fr_BE.utf8"
LC_MONETARY="fr_BE.utf8"
LC_MESSAGES="fr_BE.utf8"
LC_PAPER="fr_BE.utf8"
LC_NAME="fr_BE.utf8"
LC_ADDRESS="fr_BE.utf8"
LC_TELEPHONE="fr_BE.utf8"
LC_MEASUREMENT="fr_BE.utf8"
LC_IDENTIFICATION="fr_BE.utf8"
LC_ALL=

After running the script:

> locale
LANG=C
LANGUAGE=C
LC_CTYPE=C
LC_NUMERIC=C
LC_TIME=C
LC_COLLATE=C
LC_MONETARY=C
LC_MESSAGES=C
LC_PAPER=C
LC_NAME=C
LC_ADDRESS=C
LC_TELEPHONE=C
LC_MEASUREMENT=C
LC_IDENTIFICATION=C
LC_ALL=

[marisn]

Replying to mlennert:

Reopening this, as with a fairly fresh trunk (r65364), I get:

Please provide locale line from GRASS wx configuration file.

set_language() kicks in only if a language override is requested within wxgui settings.

[mlennert]

Replying to marisn:

Replying to mlennert:

Reopening this, as with a fairly fresh trunk (r65364), I get:

Please provide locale line from GRASS wx configuration file.

set_language() kicks in only if a language override is requested within wxgui settings.

Yep, that was it. There was an override in the wx config file.

However, this is quite confusing as I was not even using the wxgui. I don't think wxgui config should interfere with command line.

[marisn]

Hm. Probably set_language should be moved to some place to kick-in only if a GUI session is started.

Rant: The architectural problem lies in the fact that GRASS is not a monolithic app as i.e. QGIS. GUI's can be started from CLI and there are CLI apps too (and, of course, "native" widgets like "open file"!). Thus setting a language in wxgui tries to cover all potential aspects to ensure a constant user experience. The problem areas are: lack of target locale on particular system (only GRASS messages will be translated); incorrect default encoding (i.e. for Latvian language Python is certain that Win-1257 is the best thing on my UTF-8 Linux box and then hell breaks loose!); enforcing language on everything (cal; "OK" and "Cancel" buttons; all GRASS module output as it might be piped back to GUI; did I forgot something? As one can see - it is easy to mess something up and "quick fixes" need to ensure covering most of aspects as it is too easy to make things worse. (just check my own commits for language switching and you will see that it is true ;) )

[neteler]

Ticket retargeted after 7.0.1 milestone closed

[neteler]

Ticket retargeted after milestone closed

[neteler]

Ticket retargeted after milestone closed

[neteler]

Ticket retargeted after 7.0.3 milestone closed

[martinl]

Still the issue?

[mlennert]

Replying to martinl:

Still the issue?

Yes. If I set the language preference in Settings->Preferences->Appearance->Choose language to 'en', and then restart GRASS, I get 'C' locale everywhere. As Maris said, this comes from the .grass7/wx file which contains the line:

language;locale;lc_all;C

This is a GUI preference definition, but set_language() uses it for all of GRASS, including command line, even if you never start the GUI.

Either we should rename the preferences dialogues to make it clear that this is a general setting, but that it should probably not go into the wx file, or we have to change the behaviour of set_language().