winGRASS7.7.svn python3 build issues

[martinl]

WinGRASS daily builds no. 372+ fails with

  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\nviz\wxnviz.py", line
53, in <module>
    from grass.lib.vector import *
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\lib\vector.py", li
ne 340
    SF_POINT25D = 2147483649L # C:/msys64/usr/src/grass_trunk/dist.x86_64-w64-mi
ngw32/include/grass/vect/dig_defines.h: 257
                            ^
SyntaxError: invalid syntax

[martinl]

Comparing vector.py file compiled on

Windows:

'''Wrapper for vector.h

Generated with:
./ctypesgen.py --cpp gcc -E -I/c/OSGeo4W64/include -D_FILE_OFFSET_BITS=64     -I/c/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/include -I/c/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/include -D__GLIBC_HAVE_LONG_LONG -lgrass_vector.7.7.svn -IC:/OSGeo4W64/include -IC:/OSGeo4W64/include C:/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/include/grass/vector.h C:/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/include/grass/defs/vector.h C:/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/include/grass/vect/dig_structs.h C:/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/include/grass/vect/dig_defines.h C:/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/include/grass/vect/dig_externs.h -o OBJ.x86_64-w64-mingw32/vector.py

Do not modify this file.
'''
...
SF_POINT25D = 2147483649L # C:/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/include/grass/vect/dig_defines.h: 257
...

and Linux

'''Wrapper for vector.h

Generated with:
./ctypesgen.py --cpp gcc -E       -I/opt/src/grass7_trunk/dist.x86_64-pc-linux-gnu/include -I/opt/src/grass7_trunk/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG -lgrass_vector.7.7.svn -I/usr/include/postgresql -I/usr/include/gdal /opt/src/grass7_trunk/dist.x86_64-pc-linux-gnu/include/grass/vector.h /opt/src/grass7_trunk/dist.x86_64-pc-linux-gnu/include/grass/defs/vector.h /opt/src/grass7_trunk/dist.x86_64-pc-linux-gnu/include/grass/vect/dig_structs.h /opt/src/grass7_trunk/dist.x86_64-pc-linux-gnu/include/grass/vect/dig_defines.h /opt/src/grass7_trunk/dist.x86_64-pc-linux-gnu/include/grass/vect/dig_externs.h -o OBJ.x86_64-pc-linux-gnu/vector.py

Do not modify this file.
'''
...

SF_POINT25D = 2147483649 # /opt/src/grass7_trunk/dist.x86_64-pc-linux-gnu/include/grass/vect/dig_defines.h: 257
...

Any idea how to avoid L suffix?

[neteler]

---

From: Anna Petrášová <kratochanna@…> Date: Do., 13. Sep. 2018, 22:30 Subject: Re: [GRASS-user] experimental Python 3 support in trunk

On Tue, Sep 4, 2018 at 2:00 AM Markus Neteler <neteler@…> wrote: On Tue, Sep 4, 2018 at 12:39 AM Anna Petrášová <kratochanna@…> wrote:

    from grass.pygrass.vector import VectorTopo
  File "/home/mneteler/software/grass77/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/vector/__init__.py",
line 7, in <module>
    import grass.lib.vector as libvect
  File "/home/mneteler/software/grass77/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/vector.py",
line 5863
    PORT_LONG_MAX = 2147483647L
                              ^
SyntaxError: invalid syntax

regarding this issue, make sure you compile GRASS with Python 3. I get this if I compile it with Python2 and then open it with Python 3.

Anna

[hellik]

Replying to neteler:

---

From: Anna Petrášová <kratochanna@…> Date: Do., 13. Sep. 2018, 22:30 Subject: Re: [GRASS-user] experimental Python 3 support in trunk

On Tue, Sep 4, 2018 at 2:00 AM Markus Neteler <neteler@…> wrote: On Tue, Sep 4, 2018 at 12:39 AM Anna Petrášová <kratochanna@…> wrote:

    from grass.pygrass.vector import VectorTopo
  File "/home/mneteler/software/grass77/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/vector/__init__.py",
line 7, in <module>
    import grass.lib.vector as libvect
  File "/home/mneteler/software/grass77/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/vector.py",
line 5863
    PORT_LONG_MAX = 2147483647L
                              ^
SyntaxError: invalid syntax

regarding this issue, make sure you compile GRASS with Python 3. I get this if I compile it with Python2 and then open it with Python 3.

Anna

for compilation in package.sh we do:

# Force Python3
alias python=python3

adding later in the script e.g.

[...]
exec 3>&1 >> mswindows/osgeo4w/package.log 2>&1 
python --version
[...]

I get

Python 3.7.2
Sat Jan 12 15:29:17 CET 2019: STARTING dll.to.a
/c/OSGeo4W64/bin/proj_5_2.dll => mswindows/osgeo4w/lib/libproj
[...]

it seems python3 is used for compilation.

[martinl]

In 73932:

attempt to fix ctypes compilation, force python3, see #3731

[martinl]

Replying to hellik:

> alias python=python3

Defining alias does not seems to be enough. Hopefully fixed in r73932. Let's wait for build no. 373+.

[martinl]

Unfortunately last build is still no. 372. There is a new issue with building manual pages on build server

Traceback (most recent call last):
  File "C:/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/tools/g.html2man.py", line 77, in <module>
    main()
  File "C:/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/tools/g.html2man.py", line 73, in main
    outf.write(s)
  File "C:/OSGeo4W64/apps/Python37\lib\encodings\cp1250.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u2264' in position 2582: character maps to <undefined>
../../include/Make/Html.make:11: recipe for target '/c/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/docs/man/man1/r.clump.1' failed
make[4]: *** [/c/msys64/usr/src/grass_trunk/dist.x86_64-w64-mingw32/docs/man/man1/r.clump.1] Error 1

[martinl]

In 73939:

attempt to fix g.html2man wingrass python3 issue, see #3731

[martinl]

In 73940:

fix r73939 for python2, see #3731

[martinl]

Build no. 375 fails with new issues

Traceback (most recent call last):
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\wxgui.py", line 103, in OnInit
    workspace=self.workspaceFile)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\lmgr\frame.py", line 131, in __init__
    self._moduleTreeBuilder = LayerManagerModuleTree(message_handler=add_menu_error)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\lmgr\menudata.py", line 69, in __init__
    message_handler=message_handler)
  File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\menutree.py", line 72, in __init__
    xmlTree = etree.parse(filename)
  File "C:\OSGEO4~1\apps\Python37\lib\xml\etree\ElementTree.py", line 1197, in parse
    tree.parse(source, parser)
  File "C:\OSGEO4~1\apps\Python37\lib\xml\etree\ElementTree.py", line 598, in parse
    self._root = parser._parse_whole(source)
xml.etree.ElementTree.ParseError: no element found: line 1, column 0

Reason: C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\xml\module_tree_menudata.xml is empty.

Related traceback:

Traceback (most recent call last):
  File "core/toolboxes.py", line 914, in <module>
    sys.exit(main())
  File "core/toolboxes.py", line 898, in main
    userDefined=False)
  File "core/toolboxes.py", line 307, in createTree
    moduleItems=moduleItems)
  File "core/toolboxes.py", line 344, in toolboxes2menudata
    _expandRuntimeModules(root)
  File "core/toolboxes.py", line 645, in _expandRuntimeModules
    desc, keywords = _loadMetadata(name)
  File "core/toolboxes.py", line 682, in _loadMetadata
    task = gtask.parse_interface(module)
  File "C:\msys64\usr\src\grass_trunk\dist.x86_64-w64-mingw32\etc\python\grass\script\task.py", line 524, in parse_interface
    tree = etree.fromstring(get_interface_description(name))
  File "C:\msys64\usr\src\grass_trunk\dist.x86_64-w64-mingw32\etc\python\grass\script\task.py", line 476, in get_interface_description
    stderr=PIPE)
  File "C:\msys64\usr\src\grass_trunk\dist.x86_64-w64-mingw32\etc\python\grass\script\core.py", line 66, in __init__
    cmd = shutil_which(args[0])
  File "C:\msys64\usr\src\grass_trunk\dist.x86_64-w64-mingw32\etc\python\grass\script\core.py", line 222, in shutil_which
    if any(cmd.lower().endswith(ext) for ext in pathext):
  File "C:\msys64\usr\src\grass_trunk\dist.x86_64-w64-mingw32\etc\python\grass\script\core.py", line 222, in <genexpr>
    if any(cmd.lower().endswith(ext) for ext in pathext):
TypeError: endswith first arg must be str or a tuple of str, not bytes
Makefile:46: recipe for target 'xml/module_tree_menudata.xml' failed
make[4]: *** [xml/module_tree_menudata.xml] Error 1

Seems to be related to r73930 (#3723)

[martinl]

I added some debug messages around. The thumbnails.py scripts calls before g.region command the g.gisenv command(but how and where?). The problem is that once cmd is a string (should be bytes, if I am right(?) and then bytes.

g.gisenv <class 'str'>
b'g.region' <class 'bytes'>

[martinl]

In 73945:

grass.core.find_program(): force bytes, see #3731

[martinl]

Replying to martinl: the g.gisenv command(but how and where?). The problem is that once cmd is a string (should be

it was grass.core.find_program(), hopefully fixed in r73945. Let's wait for next build (no. 376+).

[martinl]

In 73956:

attempt to fix grass.task.get_interface python3 issue, see #3731

[martinl]

Great, with build no. 376 GUI starts, let's celebrate :-)

[hellik]

Replying to martinl:

Great, with build no. 376 GUI starts, let's celebrate :-)

great!

[martinl]

Reported issue seems to be solved, closing. Feel free to reopen if needed.