shutil_which() throws an exception on Linux + Python 3

[pmav99]

Traceback (most recent call last):
  File "raster/r.report/testsuite/test_r_report.py", line 68, in test_output
    self.assertModule('r.report', map='lakes', output=self.outfile)
  File "/home/feanor/Prog/git/grass-p3/repo/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py", line 1145, in assertModule
    if not shutil_which(module.name):
  File "/home/feanor/Prog/git/grass-p3/repo/dist.x86_64-pc-linux-gnu/etc/python/grass/script/core.py", line 225, in shutil_which
    name = os.path.join(encode(dir), thefile)
  File "/usr/lib/python3.7/posixpath.py", line 94, in join
    genericpath._check_arg_types('join', a, *p)
  File "/usr/lib/python3.7/genericpath.py", line 151, in _check_arg_types
    raise TypeError("Can't mix strings and bytes in path components") from None
TypeError: Can't mix strings and bytes in path components

This was introduced at ​https://trac.osgeo.org/grass/changeset/73930

A possible resolution could be:

import six

if six.PY3:
    shutil_which = shutil.which
else:
    def shutil_which(...):
        # ...

[pmav99]

shutil_which is being used by assertModule which practically means that it is not currently possible to run gunnitest based tests on Python 3.

Please prioritize this.

[neteler]

Replying to pmav99:

A possible resolution could be:

import six

if six.PY3:
    shutil_which = shutil.which
else:
    def shutil_which(...):
        # ...

Could you please provide a patch? I can also grab it from github if that's easier for you.

[pmav99]

That's the least invasive patch I can think of:

Index: lib/python/script/core.py
===================================================================
--- lib/python/script/core.py	(revision 74173)
+++ lib/python/script/core.py	(working copy)
@@ -164,7 +164,7 @@
 
     return set(cmd), scripts
 
-
+# TODO: Please replace this function with shutil.which() before 8.0 comes out
 # replacement for which function from shutil (not available in all versions)
 # from http://hg.python.org/cpython/file/6860263c05b3/Lib/shutil.py#l1068
 # added because of Python scripts running Python scripts on MS Windows
@@ -239,6 +239,8 @@
                     return name
     return None
 
+if sys.version_info.major > 2:
+    shutil_which = shutil.which
 
 # Added because of scripts calling scripts on MS Windows.
 # Module name (here cmd) differs from the file name (does not have extension).

[pmav99]

If there are not any problems with the patch, please apply it. It is really needed when testing with Python 3.

[neteler]

In 74225:

python script core: shutil_which() exception fix for Python 3 (fixes #3773)

[neteler]

Reopened with G8.0.0 milestone to remove TODO of r74225 in lib/python/script/core.py

 	167	# TODO: Please replace this function with shutil.which() before 8.0 comes out
168	168	# replacement for which function from shutil (not available in all versions)
169	169	# from http://hg.python.org/cpython/file/6860263c05b3/Lib/shutil.py#l1068