WxGUI startup screen fails if GISDBASE path contains non-latin characters

[marisn]

WXGUI startup screen goes bad when GISDBASE is pointing to folder with non-latin chars in path.

Steps to reproduce: "mkdir 'Šaursliežu dzelzceļš'" and set it as GISDBASE.

---------------------------
Error in command execution g.gisenv
---------------------------
Execution failed: 'g.gisenv set=GISDBASE=C:/Users/Maris/Documents/Šaursliežu dzelzceļš'



Details: 

Error: Unable to exectute command: 'g.gisenv set=GISDBASE=C:/Users/Maris/Documents/Šaursliežu dzelzceļš'
---------------------------
OK   
---------------------------

KŽ█DA:G_getenv(): Variable LOCATION_NAME not set
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\OSGeo4W\apps\Python25\lib\threading.py", line 486, in __bootstrap_inn
er
    self.run()
  File "C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\gcmd.py", line 52
7, in run
    shell=sys.platform=="win32")
  File "C:\Program Files\GRASS-64-SVN\Python25\lib\subprocess.py", line 594, in
__init__
    errread, errwrite)
  File "C:\Program Files\GRASS-64-SVN\Python25\lib\subprocess.py", line 816, in
_execute_child
    startupinfo)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0160' in position 9
2: ordinal not in range(128)

Exception in thread Thread-2:
Traceback (most recent call last):
  File "C:\OSGeo4W\apps\Python25\lib\threading.py", line 486, in __bootstrap_inn
er
    self.run()
  File "C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\gcmd.py", line 52
7, in run
    shell=sys.platform=="win32")
  File "C:\Program Files\GRASS-64-SVN\Python25\lib\subprocess.py", line 594, in
__init__
    errread, errwrite)
  File "C:\Program Files\GRASS-64-SVN\Python25\lib\subprocess.py", line 816, in
_execute_child
    startupinfo)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0160' in position 7
9: ordinal not in range(128)

Traceback (most recent call last):
  File "C:/Program Files/GRASS-64-SVN/etc/wxpython/gis_set.py", line 854, in <mo
dule>
    GRASSStartUp = StartUp(0)
  File "C:\OSGeo4W\apps\Python25\lib\site-packages\wx-2.8-msw-unicode\wx\_core.p
y", line 7935, in __init__
  File "C:\OSGeo4W\apps\Python25\lib\site-packages\wx-2.8-msw-unicode\wx\_core.p
y", line 7509, in _BootstrapApp
  File "C:/Program Files/GRASS-64-SVN/etc/wxpython/gis_set.py", line 825, in OnI
nit
    StartUp = GRASSStartup()
  File "C:/Program Files/GRASS-64-SVN/etc/wxpython/gis_set.py", line 165, in __i
nit__
    self._set_properties()
  File "C:/Program Files/GRASS-64-SVN/etc/wxpython/gis_set.py", line 206, in _se
t_properties
    self.OnSetDatabase(None)
  File "C:/Program Files/GRASS-64-SVN/etc/wxpython/gis_set.py", line 665, in OnS
etDatabase
    self.OnSelectLocation(None)
  File "C:/Program Files/GRASS-64-SVN/etc/wxpython/gis_set.py", line 617, in OnS
electLocation
    self.listOfLocations[self.lblocations.GetSelection()]))
  File "C:/Program Files/GRASS-64-SVN/etc/wxpython/gis_set.py", line 591, in Upd
ateMapsets
    "set=GISDBASE=%s" % self.gisdbase])
  File "C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\gcmd.py", line 35
7, in __init__
    _("Error: ") + self.GetError()))
gui_modules.gcmd.CmdError

Also interesting bit - I DON'T have such folder as "C:\OSGeo4W"

Setting this as a blocker because it's impossible to change bogous GISDBASE value as WXGUI startup screen fails to come up and provide GISDBASE choice option. Only solution is to start GRASS CLI or rm any .grass* files. Both options are not beginner-friendly.

Another part in location wizard:

access: No such file or directory
KŽ█DA:ATRAąAN┬S VIETA << C:\Users\Maris\Documents\atstar pem/mans >> nav
      pieejama
Traceback (most recent call last):
  File "C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\location_wizard.p
y", line 1710, in OnEnterPage
    self.ldatabase.SetLabel(str(database))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0160' in position 2
5: ordinal not in range(128)
Traceback (most recent call last):
  File "C:/Program Files/GRASS-64-SVN/etc/wxpython/gis_set.py", line 412, in OnW
izard
    grassdatabase = self.tgisdbase.GetValue())
  File "C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\location_wizard.p
y", line 1853, in __init__
    msg = self.OnWizFinished()
  File "C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\location_wizard.p
y", line 2011, in OnWizFinished
    set='GISDBASE=%s' % database)
  File "C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\gcmd.py", line 59
6, in RunCommand
    ps = grass.start_command(prog, flags, overwrite, quiet, verbose, **kwargs)
  File "C:\Program Files\GRASS-64-SVN\etc\python\grass\script\core.py", line 143
, in start_command
    args = make_command(prog, flags, overwrite, quiet, verbose, **options)
  File "C:\Program Files\GRASS-64-SVN\etc\python\grass\script\core.py", line 106
, in make_command
    args.append("%s=%s" % (opt, _make_val(val)))
  File "C:\Program Files\GRASS-64-SVN\etc\python\grass\script\core.py", line 73,
 in _make_val
    return str(val)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0160' in position 3
4: ordinal not in range(128)

WinGRASS-6.4.SVN-r41363-1 running Windows Vista.

[martinl]

Hopefully fixed r41390. Commands execution is affected, so please test on different platforms with various locales. Martin

[marisn]

I will test on windows in some days. Seems fine on Linux, still it's impossible to create location within such GISDBASE with location wizard. If location exists, it's possible to use it.

Traceback (most recent call last):
  File "/home/maris/soft/grass_64/dist.x86_64-unknown-linux-gnu/etc/wxpython/gui_modules/location_wizard.py", line 1712, in OnEnterPage
    self.ldatabase.SetLabel(str(database))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0160' in position 23: ordinal not in range(128)
Traceback (most recent call last):
  File "/home/maris/soft/grass_64/dist.x86_64-unknown-linux-gnu/etc/wxpython/gis_set.py", line 412, in OnWizard
    grassdatabase = self.tgisdbase.GetValue())
  File "/home/maris/soft/grass_64/dist.x86_64-unknown-linux-gnu/etc/wxpython/gui_modules/location_wizard.py", line 1855, in __init__
    msg = self.OnWizFinished()
  File "/home/maris/soft/grass_64/dist.x86_64-unknown-linux-gnu/etc/wxpython/gui_modules/location_wizard.py", line 2013, in OnWizFinished
    set='GISDBASE=%s' % database)
  File "/home/maris/soft/grass_64/dist.x86_64-unknown-linux-gnu/etc/wxpython/gui_modules/gcmd.py", line 596, in RunCommand
    ps = grass.start_command(prog, flags, overwrite, quiet, verbose, **kwargs)
  File "/home/maris/soft/grass_64/dist.x86_64-unknown-linux-gnu/etc/python/grass/script/core.py", line 143, in start_command
    args = make_command(prog, flags, overwrite, quiet, verbose, **options)
  File "/home/maris/soft/grass_64/dist.x86_64-unknown-linux-gnu/etc/python/grass/script/core.py", line 106, in make_command
    args.append("%s=%s" % (opt, _make_val(val)))
  File "/home/maris/soft/grass_64/dist.x86_64-unknown-linux-gnu/etc/python/grass/script/core.py", line 73, in _make_val
    return str(val)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0160' in position 32: ordinal not in range(128)

[martinl]

Replying to marisn:

I will test on windows in some days. Seems fine on Linux, still it's impossible to create location within such GISDBASE with location wizard. If location exists, it's possible to use it.

> Traceback (most recent call last):
>   File "/home/maris/soft/grass_64/dist.x86_64-unknown-linux-gnu/etc/wxpython/gui_modules/location_wizard.py", line 1712, in OnEnterPage
>     self.ldatabase.SetLabel(str(database))
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u0160' in position 23: ordinal not in range(128)

Please try r41393.

[marisn]

That's it! Now also location wizard works just fine with "šaursliežu dzelzceļš" :) Will perform tests on Windows Vista within this weekend.

[marisn]

Location Wizard still fails in Windows Vista WinGRASS-6.4.SVN-r41428-1:

C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\location_wizard.py:2002:
UnicodeWarning: Unicode unequal comparison failed to convert both arguments to U
nicode - interpreting them as being unequal
  if current_gdb != database:
Traceback (most recent call last):
  File "C:/Program Files/GRASS-64-SVN/etc/wxpython/gis_set.py", line 418, in OnW
izard
    grassdatabase = self.tgisdbase.GetValue())
  File "C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\location_wizard.p
y", line 1853, in __init__
    msg = self.OnWizFinished()
  File "C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\location_wizard.p
y", line 2011, in OnWizFinished
    set='GISDBASE=%s' % database)
  File "C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\gcmd.py", line 60
8, in RunCommand
    ps = grass.start_command(prog, flags, overwrite, quiet, verbose, **kwargs)
  File "c:/osgeo4w/usr/src/grass64_release/dist.i686-pc-mingw32\etc\python\grass
\script\core.py", line 145, in start_command
  File "c:/osgeo4w/usr/src/grass64_release/dist.i686-pc-mingw32\etc\python\grass
\script\core.py", line 52, in __init__
  File "C:\Program Files\GRASS-64-SVN\Python25\lib\subprocess.py", line 594, in
__init__
    errread, errwrite)
  File "C:\Program Files\GRASS-64-SVN\Python25\lib\subprocess.py", line 816, in
_execute_child
    startupinfo)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0160' in position 7
9: ordinal not in range(128)

[marisn]

Location Wizard still fails on Vista WinGRASS-6.4.SVN-r41749-1

Traceback (most recent call last):
  File "C:/Program Files/GRASS-64-SVN/etc/wxpython/gis_set.py", line 418, in OnW
izard
    grassdatabase = self.tgisdbase.GetValue())
  File "c:\osgeo4w\usr\src\grass64_release\dist.i686-pc-mingw32\etc\wxpython\gui
_modules\location_wizard.py", line 1853, in __init__
  File "c:\osgeo4w\usr\src\grass64_release\dist.i686-pc-mingw32\etc\wxpython\gui
_modules\location_wizard.py", line 2011, in OnWizFinished
  File "C:\Program Files\GRASS-64-SVN\etc\wxpython\gui_modules\gcmd.py", line 60
9, in RunCommand
    ps = grass.start_command(prog, flags, overwrite, quiet, verbose, **kwargs)
  File "c:/osgeo4w/usr/src/grass64_release/dist.i686-pc-mingw32\etc\python\grass
\script\core.py", line 145, in start_command
  File "c:/osgeo4w/usr/src/grass64_release/dist.i686-pc-mingw32\etc\python\grass
\script\core.py", line 52, in __init__
  File "C:\Program Files\GRASS-64-SVN\Python25\lib\subprocess.py", line 594, in
__init__
    errread, errwrite)
  File "C:\Program Files\GRASS-64-SVN\Python25\lib\subprocess.py", line 816, in
_execute_child
    startupinfo)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0161' in position 7
9: ordinal not in range(128)

[martinl]

I am changing priority of this bug to critical. wxGUI is not default GUI in GRASS 6.4., so wxGUI-related bugs are not blockers for this version.

[neteler]

Is this still an issue in 6.4.svn?

[neteler]

See also trac #1004

[marisn]

Replying to hellik: Don't forget to put a big, fat warning that WinGRASS works only for users with names without non-latin letters. Users migh have no write permission outside their home direcories and thus be unable to start GRASS at all.

As nightly WinGRASS server is down, I can't test SVN, but RC1 is still unable to start at all. I can't set GRASSDBASE to anything, as wxgui is failing before first screen. Still it's possible, that it's caused by #1286

[marisn]

Trying to launch GRASS 6.4 on a fresh install causes python to crash. Windows Vista 32bit. WinGRASS-6.4.SVN-r47993-1-Setup.exe Output I got by running GRASS from CMD:

C:\Program Files\GRASS 6.4.SVN>grass64svn.bat
Traceback (most recent call last):
  File "C:/Program Files/GRASS 6.4.SVN/etc/wxpython/gis_set.py", line 863, in <m
odule>
    GRASSStartUp = StartUp(0)
  File "c:\OSGeo4W\apps\Python25\lib\site-packages\wx-2.8-msw-unicode\wx\_core.p
y", line 7935, in __init__
  File "c:\OSGeo4W\apps\Python25\lib\site-packages\wx-2.8-msw-unicode\wx\_core.p
y", line 7509, in _BootstrapApp
  File "C:/Program Files/GRASS 6.4.SVN/etc/wxpython/gis_set.py", line 834, in On
Init
    StartUp = GRASSStartup()
  File "C:/Program Files/GRASS 6.4.SVN/etc/wxpython/gis_set.py", line 166, in __
init__
    self._set_properties()
  File "C:/Program Files/GRASS 6.4.SVN/etc/wxpython/gis_set.py", line 216, in _s
et_properties
    not os.path.isdir(os.path.join(self.gisdbase, location)):
  File "C:\Program Files\GRASS 6.4.SVN\Python25\lib\ntpath.py", line 67, in join

    elif isabs(b):
  File "C:\Program Files\GRASS 6.4.SVN\Python25\lib\ntpath.py", line 53, in isab
s
    s = splitdrive(s)[1]
  File "C:\Program Files\GRASS 6.4.SVN\Python25\lib\ntpath.py", line 119, in spl
itdrive
    if p[1:2] == ':':
TypeError: 'NoneType' object is unsubscriptable
KŽ█DA:G_getenv(): Variable LOCATION_NAME not set
KŽ█DA:G_getenv(): Variable LOCATION_NAME not set
KŽ█DA:G_getenv(): Variable LOCATION_NAME not set

Here's .grassrc6 file contents it creates:

GISDBASE: C:/Users/Māris
LOCATION_NAME: <UNKNOWN>
MAPSET: <UNKNOWN>

[martinl]

Replying to marisn:

As nightly WinGRASS server is down, I can't test SVN, but RC1 is still unable to start at all. I

it's not down, see [1].

[1] ​http://lists.osgeo.org/pipermail/grass-dev/2011-September/055659.html

[marisn]

Replying to martinl:

Replying to marisn:

As nightly WinGRASS server is down, I can't test SVN, but RC1 is still unable to start at all. I

it's not down, see [1].

[1] ​http://lists.osgeo.org/pipermail/grass-dev/2011-September/055659.html

Just for the record - it was down at that time: ​http://lists.osgeo.org/pipermail/grass-user/2011-March/059866.html

[mlennert]

Where are we at on this issue ? I have several students and colleagues who have been trying to use GRASS on Windows (Vista and 7) and have failed because their username contains non-latin characters.

Moritz

[marisn]

Stuck forever with 6.x releases. There is no way how to fix this issue for 6.x. Only hope is to move away from anything shell related in 7.x and thus avoid any OEM codepage issues on Windows.

Solution for end-users: create a new user with only latin letters in it's name for GRASS usage.

See more here: ​http://lists.osgeo.org/pipermail/grass-dev/2011-March/053874.html

Replying to mlennert:

Where are we at on this issue ? I have several students and colleagues who have been trying to use GRASS on Windows (Vista and 7) and have failed because their username contains non-latin characters.

Moritz

[mlennert]

Any news on this issue, at least for GRASS 7 ?

Moritz

[mlennert]

Replying to mlennert:

Any news on this issue, at least for GRASS 7 ?

I've just tested in WinXP and can reproduce the startup error when the path to GISDBASE contains special characters (in this case latin1 accents).

If I follow Maris' comment:18 then the only solution would be to call grass.py directly from a python script, and not from a .bat. Is that correct ?

[marisn]

Replying to mlennert:

Replying to mlennert:

Any news on this issue, at least for GRASS 7 ?

If I follow Maris' comment:18 then the only solution would be to call grass.py directly from a python script, and not from a .bat. Is that correct ?

I just tested GRASS 7 on Vista. It seems to pass the "welcome" screen just fine, still CLI (CMD) is not set up correctly (LOCATION_NAME not set). g.gui starts fine, still it is not possible to import any data as vector import wizard fails to see any layers in a shapefile and raster data importing fails when calling r.in.gdal with "file doesn't exist" error. Exiting from CMD also fails with "Failed to start shell 'C:\Program Files\GRASS GIS 7.0.svn\msys\bin\sh.exe'".

My guess - elimination of CMD part on windows as per #1891 AND elimination of touching shell in ANY way is the only solution to fix this issue.

[martinl]

Replying to marisn:

I just tested GRASS 7 on Vista. It seems to pass the "welcome" screen just fine, still CLI (CMD) is not set up correctly (LOCATION_NAME not set). g.gui starts fine, still it is not possible to import any data as vector import wizard fails to see any layers in a shapefile and raster data importing fails when calling r.in.gdal with "file doesn't exist" error. Exiting from CMD also fails with "Failed to start shell 'C:\Program Files\GRASS GIS 7.0.svn\msys\bin\sh.exe'".

My guess - elimination of CMD part on windows as per #1891 AND elimination of touching shell in ANY way is the only solution to fix this issue.

please could you do the test again with most recent GRASS 7.0 version?

[marisn]

Replying to martinl:

please could you do the test again with most recent GRASS 7.0 version?

Yes, startup screen now works fine. Still it has just uncovered next set of problems. As this issue is fixed, I am closing it and will open new ones for other failures.