Opened 12 years ago
Closed 6 years ago
#1672 closed defect (fixed)
gcpmanager fails to list source mapsets if non-ascii characters in grass data dir path
Reported by: | hamish | Owned by: | |
---|---|---|---|
Priority: | normal | Milestone: | 7.2.0 |
Component: | wxGUI | Version: | svn-trunk |
Keywords: | georectifier, UnicodeDecodeError | Cc: | torsti.schulz@… |
CPU: | x86-64 | Platform: | Linux |
Description
[Hi, one fwd'd from the Debian bug tracker, http://bugs.debian.org/676190 --H]
From: Torsti Schulz <torsti.schulz at iki dot fi> To: Debian Bug Tracking System Subject: grass-gui: gcpmanager fails to list source mapsets if non-ascii characters in grass data dir path Date: Tue, 05 Jun 2012 14:35:48 +0300 Package: grass-gui Version: 6.4.2-1 Severity: normal Hi, Using the wxpython GUI Georectify tool is impossible if the path to the grass data directory (e.g. GRASSDATA) contains non ascii characters (e.g. '/home/user/Työpöytä/GRASSDATA'). Selecting the source location works, but the 'Select source mapset:' dropdown menu remains greyed out. The command console shows the following error message, after selecting the source location: *** Traceback (most recent call last): File "/usr/lib/grass64/etc/wxpython/gui_modules/gcpmanager.py", line 377, in OnLocation tmplist = os.listdir(os.path.join(self.grassdatabase, self.xylocation)) File "/usr/lib/python2.7/posixpath.py", line 71, in join path += '/' + b UnicodeDecodeError : 'ascii' codec can't decode byte 0xc3 in position 15: ordinal not in range(128) *** Changing the grass data directory path to only contain ascii characters (e.g. '/home/user/GRASSDATA') makes the Georectify tool usable. This seems to an issue with how the python code handles path names and unicode characters. Cheers, Torsti -- System Information: Debian Release: wheezy/sid APT prefers testing APT policy: (700, 'testing') Architecture: amd64 (x86_64) Kernel: Linux 3.2.0-2-amd64 (SMP w/4 CPU cores) Locale: LANG=fi_FI.utf8, LC_CTYPE=fi_FI.utf8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages grass-gui depends on: ii gnome-terminal [x-terminal-emulator] 3.4.1.1-1 ii grass-core 6.4.2-1 ii lesstif2 1:0.95.2-1 ii libc6 2.13-33 ii libfreetype6 2.4.9-1 ii libgdal1 1.9.0-2 ii libgl1-mesa-glx [libgl1] 8.0.2-2 ii libglu1-mesa [libglu1] 8.0.2-2 ii libice6 2:1.0.8-2 ii libpng12-0 1.2.49-1 ii libsm6 2:1.2.1-2 ii libtiff5 4.0.1-7 ii libx11-6 2:1.4.99.901-2 ii libxext6 2:1.3.1-2 ii libxmu6 2:1.1.1-1 ii libxt6 1:1.1.3-1 ii python 2.7.2-10 ii python-opengl 3.0.1-1 ii python-wxgtk2.8 2.8.12.1-11 ii tcl8.5 8.5.11-2 ii tk8.5 8.5.11-2 ii xterm [x-terminal-emulator] 278-1 ii zlib1g 1:1.2.7.dfsg-11 Versions of packages grass-gui recommends: ii ghostscript 9.05~dfsg-6 ii python-imaging 1.1.7-4 grass-gui suggests no packages.
Change History (15)
comment:1 by , 12 years ago
comment:2 by , 12 years ago
Version: | 6.4.2 → svn-trunk |
---|
This one is still around (r55643). It can be traced back to 'self.grassdatabase' in 'OnLocation' (gui/wxpython/gcp/manager.py, line 376), which gets its value at line 94:
self.grassdatabase = grass.gisenv()['GISDBASE']
'gisenv' is defined at line 720 in lib/python/script/core.py:
s = read_command("g.gisenv", flags='n') return parse_key_val(s)
Neither the string 's' read from g.gisenv nor the dictionary returned by 'parse_key_val' is explicitly decoded to Unicode, so when it is finally implicitly cast to Unicode somewhere in Python's bowels, the default ascii codec is used, which leads to errors if $GISDBASE contains non ascii characters.
This a wider python (at least for < 3.x) problem, and fixing it for this specific issue won't probably solve all existing UnicodeDecodeErrors nor prevent new ones from surfacing, but at least any GUI component calling gisenv can probably be fixed at one location.
Any filenames/paths read into wxpython/pygrass should probably be explicitly decoded to Unicode, e.g. with the 'decode()' function defined at line 79 in script/core.py. Enforcing only ascii chars works for mapset and location names, but can't/shouldn't be the solution for path components not under GRASS's direct control.
follow-up: 8 comment:3 by , 9 years ago
Milestone: | 6.4.3 → 7.0.2 |
---|
As reported in Debian Bug #676190, this issue is still present in GRASS 7.0.1. When non-ascii characters are in the grass data path, the 'Select source mapset:' dropdown is populated with empty options. When no non-ascii characters are the path the dropdown is populated with the expected mapsets.
comment:5 by , 9 years ago
This seems to be fixed in trunk, but not in release70. However, I don't have the time right now to try and find out which of the fixes in trunk fixed this particular issue...
follow-up: 9 comment:8 by , 9 years ago
Replying to sebastic:
As reported in Debian Bug #676190, this issue is still present in GRASS 7.0.1. When non-ascii characters are in the grass data path, the 'Select source mapset:' dropdown is populated with empty options. When no non-ascii characters are the path the dropdown is populated with the expected mapsets.
Anybody able to identify the change we need to backport?
follow-up: 10 comment:9 by , 9 years ago
Replying to neteler:
Replying to sebastic:
As reported in Debian Bug #676190, this issue is still present in GRASS 7.0.1. When non-ascii characters are in the grass data path, the 'Select source mapset:' dropdown is populated with empty options. When no non-ascii characters are the path the dropdown is populated with the expected mapsets.
Anybody able to identify the change we need to backport?
r65787 ? It adds encoding and decoding to the parse_key_val which is used in gisenv(). I'm still too confused by encoding handling in Python to be able to say whether this is the "right" solution to the problem, though.
comment:10 by , 9 years ago
Replying to mlennert:
Replying to neteler:
Replying to sebastic:
As reported in Debian Bug #676190, this issue is still present in GRASS 7.0.1. When non-ascii characters are in the grass data path, the 'Select source mapset:' dropdown is populated with empty options. When no non-ascii characters are the path the dropdown is populated with the expected mapsets.
Anybody able to identify the change we need to backport?
r65787 ? It adds encoding and decoding to the parse_key_val which is used in gisenv().
Yes, by merging r65787 the bug seems to be fixed.
I'm still too confused by encoding handling in Python to be able to say whether this is the "right" solution to the problem, though.
Could some of the wxgui gurus look at this to see if this can just be backported or if another solution would be better ?
comment:11 by , 8 years ago
Milestone: | 7.0.4 → 7.0.5 |
---|
comment:12 by , 8 years ago
Milestone: | 7.0.5 → 7.0.6 |
---|
comment:13 by , 8 years ago
While backporting the fix to 7.0.x would be nice, it is no longer present in 7.2.0RC2 (nor RC1).
comment:14 by , 7 years ago
Milestone: | 7.0.6 → 7.0.7 |
---|
comment:15 by , 6 years ago
Milestone: | 7.0.7 → 7.2.0 |
---|---|
Resolution: | → fixed |
Status: | new → closed |
see also
and
Hamish