winGRASS7.7.svn - core.py - python 3 issues

[hellik]

tested with a self compiled winGRASS7.7.svn with python 3

C:\>g.version -e -g
version=7.7.svn
date=2019
revision=r73896M
build_date=2019-01-01
build_platform=x86_64-w64-mingw32
build_off_t_size=8
proj4=5.2.0
gdal=2.4.0
geos=3.7.0
sqlite=3.26.0

C:\>grass77svn --text
Starting GRASS GIS...
WARNING: Concurrent mapset locking is not supported on Windows
Cleaning up temporary files...

          __________  ___   __________    _______________
         / ____/ __ \/   | / ___/ ___/   / ____/  _/ ___/
        / / __/ /_/ / /| | \__ \\_  \   / / __ / / \__ \
       / /_/ / _, _/ ___ |___/ /__/ /  / /_/ // / ___/ /
       \____/_/ |_/_/  |_/____/____/   \____/___//____/

Welcome to GRASS GIS 7.7.svn (r73896M)
GRASS GIS homepage:                      https://grass.osgeo.org
This version running through:            Command Prompt (C:\WINDOWS\system32\cmd.exe)
Help is available with the command:      g.manual -i
See the licence terms with:              g.version -c
See citation options with:               g.version -x
Start the GUI with:                      g.gui wxpython
When ready to quit enter:                exit

Microsoft Windows [Version 10.0.17763.195]
(c) 2018 Microsoft Corporation. Alle Rechte vorbehalten.

C:\>python
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

then g.gui wxpython errors in

C:\>g.gui wxpython
Launching <wxpython> GUI in the background, please wait...
Traceback (most recent call last):
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 41, in <module>
    import __builtin__
ModuleNotFoundError: No module named '__builtin__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn/gui/wxpython/wxgui.py", line 27, in <module>
    from core import globalvar
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\globalvar.py", line 31, in <module>
    from core.debug import Debug
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py", line 25, in <module>
    import grass.script as grass
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\__init__.py", line 5, in <module>
    from .core   import *
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 47, in <module>
    from os import environb as environ
ImportError: cannot import name 'environb' from 'os' (C:\OSGEO4~1\apps\Python37\lib\os.py)

it seems that core.py hangs around

try:
    # python2
    import __builtin__
    from os import environ
    __builtin__.__dict__['_'] = __builtin__.__dict__['_'].__self__.ugettext
except ImportError:
    # python3
    import builtins as __builtin__
    from os import environb as environ
    unicode = str
    __builtin__.__dict__['_'] = __builtin__.__dict__['_'].__self__.gettext

[hellik]

Replying to hellik:

from os import environb as environ

ImportError: cannot import name 'environb' from 'os' (C:\OSGEO4~1\apps\Python37\lib\os.py) }}}

regarding the environb issue, see

​https://docs.python.org/3/library/os.html

 os.environb¶

    Bytes version of environ: a mapping object representing the environment as byte strings. environ and environb are synchronized (modify environb updates environ, and vice versa).

    environb is only available if supports_bytes_environ is True.

​https://docs.python.org/3/library/os.html#os.supports_bytes_environ

 os.supports_bytes_environ

    True if the native OS type of the environment is bytes (eg. False on Windows).

it seems this isn't available in windows

[hellik]

Replying to hellik:

Replying to hellik:

from os import environb as environ

ImportError: cannot import name 'environb' from 'os' (C:\OSGEO4~1\apps\Python37\lib\os.py) }}}

regarding the environb issue, see

​https://docs.python.org/3/library/os.html

 os.environb¶

    Bytes version of environ: a mapping object representing the environment as byte strings. environ and environb are synchronized (modify environb updates environ, and vice versa).

    environb is only available if supports_bytes_environ is True.

​https://docs.python.org/3/library/os.html#os.supports_bytes_environ

 os.supports_bytes_environ

    True if the native OS type of the environment is bytes (eg. False on Windows).

it seems this isn't available in windows

see (GRASS related):

​https://stackoverflow.com/questions/52269281/fix-import-error-on-using-environb-in-python#

[martinl]

In 73906:

avoid usage of os.environb, inspired by ​https://stackoverflow.com/questions/52269281/fix-import-error-on-using-environb-in-python, see #3723

[martinl]

Replying to hellik:

​https://stackoverflow.com/questions/52269281/fix-import-error-on-using-environb-in-python#

suggested modification applied in r73906 (for better testing). It will appear in winGRASS build no. 364+.

[hellik]

Replying to martinl:

Replying to hellik:

​https://stackoverflow.com/questions/52269281/fix-import-error-on-using-environb-in-python#

suggested modification applied in r73906 (for better testing). It will appear in winGRASS build no. 364+.

tested with

Welcome to GRASS GIS 7.7.svn (r73906)

C:\>g.gui wxpython
Launching <wxpython> GUI in the background, please wait...

C:\>Traceback (most recent call last):
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn/gui/wxpython/wxgui.py", line 27, in
<module>
    from core import globalvar
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\gui\wxpython\core\globalvar.py", lin
e 31, in <module>
    from core.debug import Debug
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py", line 77
, in <module>
    Debug = DebugMsg()
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py", line 39
, in __init__
    self.SetLevel()
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py", line 45
, in SetLevel
    self.debuglevel = int(grass.gisenv().get('WX_DEBUG', 0))
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 1082, in gisenv
    s = read_command("g.gisenv", flags='n', env=env)
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 502, in read_command
    process = pipe_command(*args, **kwargs)
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 471, in pipe_command
    return start_command(*args, **kwargs)
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 396, in start_command
    if debug_level() > 0:
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 1670, in debug_level
    _debug_level = int(gisenv().get('DEBUG', 0))
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 1082, in gisenv
    s = read_command("g.gisenv", flags='n', env=env)
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 502, in read_command
    process = pipe_command(*args, **kwargs)
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 471, in pipe_command
    return start_command(*args, **kwargs)
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 401, in start_command
    return Popen(args, **popts)
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 66, in __init__
    cmd = shutil_which(args[0])
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 220, in shutil_which
    if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
  File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 220, in <genexpr>
    if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
TypeError: endswith first arg must be bytes or a tuple of bytes, not str

[hellik]

Replying to hellik:

[...]

if any(cmd.lower().endswith(ext.lower()) for ext in pathext):

File "C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li

ne 220, in <genexpr>

if any(cmd.lower().endswith(ext.lower()) for ext in pathext):

TypeError: endswith first arg must be bytes or a tuple of bytes, not str }}}

now also tested with the OSGeo4W 64bit version.

C:\>g.version -g
version=7.7.svn
date=2019
revision=r73906
build_date=2019-01-04
build_platform=x86_64-w64-mingw32
build_off_t_size=8

same error

C:\>g.gui wxpython
Launching <wxpython> GUI in the background, please wait...
Traceback (most recent call last):
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn/gui/wxpython/wxgui.py", line 27, in <module>
    from core import globalvar
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\globalvar.py", line 31, in <module>
    from core.debug import Debug
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py", line 77, in <module>
    Debug = DebugMsg()
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py", line 39, in __init__
    self.SetLevel()
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py", line 45, in SetLevel
    self.debuglevel = int(grass.gisenv().get('WX_DEBUG', 0))
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 1082, in gisenv
    s = read_command("g.gisenv", flags='n', env=env)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 502, in read_command
    process = pipe_command(*args, **kwargs)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 471, in pipe_command
    return start_command(*args, **kwargs)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 396, in start_command
    if debug_level() > 0:
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 1670, in debug_level
    _debug_level = int(gisenv().get('DEBUG', 0))
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 1082, in gisenv
    s = read_command("g.gisenv", flags='n', env=env)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 502, in read_command
    process = pipe_command(*args, **kwargs)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 471, in pipe_command
    return start_command(*args, **kwargs)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 401, in start_command
    return Popen(args, **popts)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 66, in __init__
    cmd = shutil_which(args[0])
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 220, in shutil_which
    if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", line 220, in <genexpr>
    if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
TypeError: endswith first arg must be bytes or a tuple of bytes, not str

[martinl]

In 73930:

wingrass77 core python3 issues, see #3723

[martinl]

Reported issue should be fixed in r73930. You can test it with WinGRASS daily build no. 372+. See related #3731

[martinl]

Build no.372 tested. Original issue seems to be fixed. Feel free to reopen if needed.

[martinl]

Build no. 375 fails with new issues

Traceback (most recent call last):
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\wxgui.py", line 103, in OnInit
    workspace=self.workspaceFile)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\lmgr\frame.py", line 131, in __init__
    self._moduleTreeBuilder = LayerManagerModuleTree(message_handler=add_menu_error)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\lmgr\menudata.py", line 69, in __init__
    message_handler=message_handler)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\menutree.py", line 72, in __init__
    xmlTree = etree.parse(filename)
  File "C:\OSGEO4~1\apps\Python37\lib\xml\etree\ElementTree.py", line 1197, in parse
    tree.parse(source, parser)
  File "C:\OSGEO4~1\apps\Python37\lib\xml\etree\ElementTree.py", line 598, in parse
    self._root = parser._parse_whole(source)
xml.etree.ElementTree.ParseError: no element found: line 1, column 0

Reason: C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\xml\module_tree_menudata.xml is empty.

Related traceback:

Traceback (most recent call last):
  File "core/toolboxes.py", line 914, in <module>
    sys.exit(main())
  File "core/toolboxes.py", line 898, in main
    userDefined=False)
  File "core/toolboxes.py", line 307, in createTree
    moduleItems=moduleItems)
  File "core/toolboxes.py", line 344, in toolboxes2menudata
    _expandRuntimeModules(root)
  File "core/toolboxes.py", line 645, in _expandRuntimeModules
    desc, keywords = _loadMetadata(name)
  File "core/toolboxes.py", line 682, in _loadMetadata
    task = gtask.parse_interface(module)
  File "C:\msys64\usr\src\grass_trunk\dist.x86_64-w64-mingw32\etc\python\grass\script\task.py", line 524, in parse_interface
    tree = etree.fromstring(get_interface_description(name))
  File "C:\msys64\usr\src\grass_trunk\dist.x86_64-w64-mingw32\etc\python\grass\script\task.py", line 476, in get_interface_description
    stderr=PIPE)
  File "C:\msys64\usr\src\grass_trunk\dist.x86_64-w64-mingw32\etc\python\grass\script\core.py", line 66, in __init__
    cmd = shutil_which(args[0])
  File "C:\msys64\usr\src\grass_trunk\dist.x86_64-w64-mingw32\etc\python\grass\script\core.py", line 222, in shutil_which
    if any(cmd.lower().endswith(ext) for ext in pathext):
  File "C:\msys64\usr\src\grass_trunk\dist.x86_64-w64-mingw32\etc\python\grass\script\core.py", line 222, in <genexpr>
    if any(cmd.lower().endswith(ext) for ext in pathext):
TypeError: endswith first arg must be str or a tuple of str, not bytes
Makefile:46: recipe for target 'xml/module_tree_menudata.xml' failed
make[4]: *** [xml/module_tree_menudata.xml] Error 1

Seems to be related to r73930 (#3723)