Unicode unequal comparison in location wizard when title contains non-ascii letters

[marisn]

In location wizard one just has to use "āšņļ" as the location title.

C:\Program Files\GRASS GIS 7.0.svn\etc\gui\wxpython\location_wizard\wizard.py:20
64: UnicodeWarning: Unicode unequal comparison failed to convert both arguments
to Unicode - interpreting them as being unequal
  if current_gdb != database:

PERMANENT/MYNAME seems to be created correctly (UTF-8 without BOM).

Tested on Vista with r59159

[marisn]

Still happens with 7.0.0 r63925 on Vista.

[hellik]

Replying to marisn:

Still happens with 7.0.0 r63925 on Vista.

location title works for me in Vista, but not location name:

Traceback (most recent call last):
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\gui\wxpython\gui_core\widgets.py", l
ine 759, in Validate
    if not self._condition(text):
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", li
ne 1509, in legal_name
    's': s, 'il': illegal})
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", li
ne 570, in warning
    message(msg, flag='w')
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", li
ne 516, in message
    run_command("g.message", flags=flag, message=msg, errors='ignore')
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", li
ne 373, in run_command
    ps = start_command(*args, **kwargs)
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", li
ne 349, in start_command
    val = encode(val)
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\utils.py", l
ine 218, in encode
    return string.encode(enc)
  File "C:\OSGeo4W\apps\Python27\lib\encodings\cp1252.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u0101' in position
 18: character maps to <undefined>

[annakrat]

Replying to marisn:

In location wizard one just has to use "āšņļ" as the location title.

C:\Program Files\GRASS GIS 7.0.svn\etc\gui\wxpython\location_wizard\wizard.py:20
64: UnicodeWarning: Unicode unequal comparison failed to convert both arguments
to Unicode - interpreting them as being unequal
  if current_gdb != database:

PERMANENT/MYNAME seems to be created correctly (UTF-8 without BOM).

Tested on Vista with r59159

It doesn't seem to do anything with the location title, the location name is important. What is the name of the location you are creating and what is the name of the location you were previously in (saved in rc file I think)?

[glynn]

Replying to hellik:

  File "C:\OSGeo4W\apps\Python27\lib\encodings\cp1252.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u0101' in position
 18: character maps to <undefined>

No surprise there. U+0101 ("LATIN SMALL LETTER A WITH MACRON") doesn't exist in cp1252 (Western Europe); I believe that cp1257 (Baltic) is the correct codepage for this.

[marisn]

Re-tested with 7.0.0 r64041 after removing the AppData/Roaming/GRASS folder and it was still an issue. I couldn't perform any other tests, as g.gui is failing on this build (different issue).

I managed to track it down - grass.gisenv() returns str, but database is unicode. I took the liberty to fix it for 7.1 in r64044 No more warnings on my Vista box and Linux box.

I would say that this needs to be backported to releasebranch (unless it needs to be reverted as a band aid and grass.gisenv() is changed to return unicode strings). I'm leaving for the wx and Python gurus to decide.

[neteler]

The backport still hasn't happened...

[neteler]

Is this working meanwhile or is the backport of r64044 needed/useful?

[hellik]

tested with

GRASS version: 7.1.svn                                                          
GRASS SVN revision: 67403                                                       
Build date: 2015-12-29                                                          
Build platform: i386-w64-mingw32                                                
GDAL: 1.11.3                                                                    
PROJ.4: 4.9.2                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.7.17                                                                  
Python: 2.7.4                                                                   
wxPython: 2.8.12.1                                                              
Platform: Windows-7-6.1.7601-SP1 (OSGeo4W) 

and āšņļ as the location name.

Traceback (most recent call last):
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\gui\wxpython\gui_core\widgets.py",
line 765, in Validate
    if not self._condition(text):
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", l
ine 1559, in legal_name
    's': s, 'il': illegal})
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", l
ine 618, in warning
    message(msg, flag='w')
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", l
ine 563, in message
    run_command("g.message", flags=flag, message=msg, errors='ignore')
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", l
ine 408, in run_command
    ps = start_command(*args, **kwargs)
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", l
ine 367, in start_command
    val = encode(val)
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\utils.py",
line 174, in encode
    return string.encode(enc)
  File "C:\OSGEO4~2\apps\Python27\lib\encodings\cp1252.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u0101' in position
 18: character maps to <undefined>

so still an issue?

[marisn]

Replying to hellik:

so still an issue?

Your test is not an indicator of THIS problem. As per Glynn's comment, "āšņļ" will be a valid test only for Windows set to run with Baltic locale, you most likely run Western Europe version thus the test text should be: "äêíõû"

As in one year nobody has cared to deal with encoding mess, closing this as fixed in 7.1.0

[annakrat]

Replying to marisn:

Replying to hellik:

so still an issue?

Your test is not an indicator of THIS problem. As per Glynn's comment, "āšņļ" will be a valid test only for Windows set to run with Baltic locale, you most likely run Western Europe version thus the test text should be: "äêíõû"

As in one year nobody has cared to deal with encoding mess, closing this as fixed in 7.1.0

I am not getting any error in trunk or releasebranch. When I specify new location name the wizard correctly reports non-ascii characters and doesn't allow me to continue. It's on my Ubuntu. Is there anything specific to trigger the error?

[neteler]

Milestone renamed