Opened 6 years ago

Closed 4 years ago

Last modified 4 years ago

#2205 closed defect (fixed)

Unicode unequal comparison in location wizard when title contains non-ascii letters

Reported by: marisn Owned by: grass-dev@…
Priority: normal Milestone: 7.2.0
Component: wxGUI Version: svn-trunk
Keywords: Cc:
CPU: Unspecified Platform: Unspecified

Description

In location wizard one just has to use "āšņļ" as the location title.

C:\Program Files\GRASS GIS 7.0.svn\etc\gui\wxpython\location_wizard\wizard.py:20
64: UnicodeWarning: Unicode unequal comparison failed to convert both arguments
to Unicode - interpreting them as being unequal
  if current_gdb != database:

PERMANENT/MYNAME seems to be created correctly (UTF-8 without BOM).

Tested on Vista with r59159

Change History (11)

comment:1 Changed 5 years ago by marisn

Still happens with 7.0.0 r63925 on Vista.

comment:2 in reply to:  1 ; Changed 5 years ago by hellik

Replying to marisn:

Still happens with 7.0.0 r63925 on Vista.

location title works for me in Vista, but not location name:

Traceback (most recent call last):
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\gui\wxpython\gui_core\widgets.py", l
ine 759, in Validate
    if not self._condition(text):
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", li
ne 1509, in legal_name
    's': s, 'il': illegal})
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", li
ne 570, in warning
    message(msg, flag='w')
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", li
ne 516, in message
    run_command("g.message", flags=flag, message=msg, errors='ignore')
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", li
ne 373, in run_command
    ps = start_command(*args, **kwargs)
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", li
ne 349, in start_command
    val = encode(val)
  File "C:\OSGeo4W\apps\grass\grass-7.1.svn\etc\python\grass\script\utils.py", l
ine 218, in encode
    return string.encode(enc)
  File "C:\OSGeo4W\apps\Python27\lib\encodings\cp1252.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u0101' in position
 18: character maps to <undefined>

comment:3 in reply to:  description Changed 5 years ago by annakrat

Replying to marisn:

In location wizard one just has to use "āšņļ" as the location title.

C:\Program Files\GRASS GIS 7.0.svn\etc\gui\wxpython\location_wizard\wizard.py:20
64: UnicodeWarning: Unicode unequal comparison failed to convert both arguments
to Unicode - interpreting them as being unequal
  if current_gdb != database:

PERMANENT/MYNAME seems to be created correctly (UTF-8 without BOM).

Tested on Vista with r59159

It doesn't seem to do anything with the location title, the location name is important. What is the name of the location you are creating and what is the name of the location you were previously in (saved in rc file I think)?

comment:4 in reply to:  2 Changed 5 years ago by glynn

Replying to hellik:

  File "C:\OSGeo4W\apps\Python27\lib\encodings\cp1252.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u0101' in position
 18: character maps to <undefined>

No surprise there. U+0101 ("LATIN SMALL LETTER A WITH MACRON") doesn't exist in cp1252 (Western Europe); I believe that cp1257 (Baltic) is the correct codepage for this.

comment:5 Changed 5 years ago by marisn

Re-tested with 7.0.0 r64041 after removing the AppData/Roaming?/GRASS folder and it was still an issue. I couldn't perform any other tests, as g.gui is failing on this build (different issue).

I managed to track it down - grass.gisenv() returns str, but database is unicode. I took the liberty to fix it for 7.1 in r64044 No more warnings on my Vista box and Linux box.

I would say that this needs to be backported to releasebranch (unless it needs to be reverted as a band aid and grass.gisenv() is changed to return unicode strings). I'm leaving for the wx and Python gurus to decide.

comment:6 Changed 4 years ago by neteler

Milestone: 7.0.07.0.3

The backport still hasn't happened...

comment:7 Changed 4 years ago by neteler

Is this working meanwhile or is the backport of r64044 needed/useful?

comment:8 Changed 4 years ago by hellik

tested with

GRASS version: 7.1.svn                                                          
GRASS SVN revision: 67403                                                       
Build date: 2015-12-29                                                          
Build platform: i386-w64-mingw32                                                
GDAL: 1.11.3                                                                    
PROJ.4: 4.9.2                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.7.17                                                                  
Python: 2.7.4                                                                   
wxPython: 2.8.12.1                                                              
Platform: Windows-7-6.1.7601-SP1 (OSGeo4W) 

and āšņļ as the location name.

Traceback (most recent call last):
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\gui\wxpython\gui_core\widgets.py",
line 765, in Validate
    if not self._condition(text):
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", l
ine 1559, in legal_name
    's': s, 'il': illegal})
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", l
ine 618, in warning
    message(msg, flag='w')
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", l
ine 563, in message
    run_command("g.message", flags=flag, message=msg, errors='ignore')
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", l
ine 408, in run_command
    ps = start_command(*args, **kwargs)
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\core.py", l
ine 367, in start_command
    val = encode(val)
  File "C:\OSGEO4~2\apps\grass\grass-7.1.svn\etc\python\grass\script\utils.py",
line 174, in encode
    return string.encode(enc)
  File "C:\OSGEO4~2\apps\Python27\lib\encodings\cp1252.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u0101' in position
 18: character maps to <undefined>

so still an issue?

Last edited 4 years ago by hellik (previous) (diff)

comment:9 in reply to:  8 ; Changed 4 years ago by marisn

Milestone: 7.0.37.1.0
Resolution: fixed
Status: newclosed

Replying to hellik:

so still an issue?

Your test is not an indicator of THIS problem. As per Glynn's comment, "āšņļ" will be a valid test only for Windows set to run with Baltic locale, you most likely run Western Europe version thus the test text should be: "äêíõû"

As in one year nobody has cared to deal with encoding mess, closing this as fixed in 7.1.0

comment:10 in reply to:  9 Changed 4 years ago by annakrat

Replying to marisn:

Replying to hellik:

so still an issue?

Your test is not an indicator of THIS problem. As per Glynn's comment, "āšņļ" will be a valid test only for Windows set to run with Baltic locale, you most likely run Western Europe version thus the test text should be: "äêíõû"

As in one year nobody has cared to deal with encoding mess, closing this as fixed in 7.1.0

I am not getting any error in trunk or releasebranch. When I specify new location name the wizard correctly reports non-ascii characters and doesn't allow me to continue. It's on my Ubuntu. Is there anything specific to trigger the error?

comment:11 Changed 4 years ago by neteler

Milestone: 7.1.07.2.0

Milestone renamed

Note: See TracTickets for help on using tickets.