Python 3 Error using MultiModule and ParallelModuleQueue

[AnikaBettge]

I tested the examples on https://grass.osgeo.org/grass77/manuals/libpython/pygrass.modules.interface.html?highlight=parallelmodulequeue#pygrass.modules.interface.module.ParallelModuleQueue with Python 3.6.7.

The example "Check MultiModule approach with three by two processes running in a background process" leads to

...

Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 235, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib/python3.6/multiprocessing/reduction.py", line 52, in dumps
    cls(buf, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects

...

The same error appears in the MultiModule example "Asynchronous module run, setting finish = False and using temporary region".

Using python 2.7.15rc1 all works fine.

[martinl]

Ticket retargeted after milestone closed

[neteler]

Does the suggestion here apply?

​https://stackoverflow.com/questions/44144584/typeerror-cant-pickle-thread-lock-objects

[neteler]

Python test script

[neteler]

Still broken... (test script attached):

GRASS 7.8.2dev (nc_spm_08):~ > python3 test_multimodule.py 
projection: 99 (Lambert Conformal Conic)
zone:       0
datum:      nad83
ellipsoid:  a=6378137 es=0.006694380022900787
north:      228500
south:      215000
west:       630000
east:       645000
nsres:      10
ewres:      10
rows:       1350
cols:       1500
cells:      2025000
projection: 99 (Lambert Conformal Conic)
zone:       0
datum:      nad83
ellipsoid:  a=6378137 es=0.006694380022900787
north:      228500
south:      215000
west:       630000
east:       645000
nsres:      10
ewres:      10
rows:       1350
cols:       1500
cells:      2025000
projection: 99 (Lambert Conformal Conic)
zone:       0
datum:      nad83
ellipsoid:  a=6378137 es=0.006694380022900787
north:      228500
south:      215000
west:       630000
east:       645000
nsres:      10
ewres:      10
rows:       1350
cols:       1500
cells:      2025000
 100%
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib64/python3.7/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects
 100%
 100%
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib64/python3.7/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib64/python3.7/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects

[sbl]

This is fixed in master. See: ​https://github.com/OSGeo/grass/pull/1407 and ​https://lists.osgeo.org/pipermail/grass-dev/2021-May/095184.html

It has not (yet) been backported to 7.8 though due to an API change...

Can we close this or is backporting required (with GRASS 8 containing the fix on its way....)?