Opened 5 years ago

Last modified 2 years ago

#3799 new defect

Python 3 Error using MultiModule and ParallelModuleQueue

Reported by: AnikaBettge Owned by: grass-dev@…
Priority: normal Milestone: 7.8.3
Component: Python Version: git-releasebranch78
Keywords: python3, multimodule Cc:
CPU: Unspecified Platform: Linux

Description

I tested the examples on https://grass.osgeo.org/grass77/manuals/libpython/pygrass.modules.interface.html?highlight=parallelmodulequeue#pygrass.modules.interface.module.ParallelModuleQueue with Python 3.6.7.

The example "Check MultiModule approach with three by two processes running in a background process" leads to

...

Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 235, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib/python3.6/multiprocessing/reduction.py", line 52, in dumps
    cls(buf, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects

...

The same error appears in the MultiModule example "Asynchronous module run, setting finish = False and using temporary region".

Using python 2.7.15rc1 all works fine.

Attachments (1)

test_multimodule.py (609 bytes ) - added by neteler 4 years ago.
Python test script

Download all attachments as: .zip

Change History (5)

comment:1 by martinl, 5 years ago

Milestone: 7.6.17.6.2

Ticket retargeted after milestone closed

by neteler, 4 years ago

Attachment: test_multimodule.py added

Python test script

comment:3 by neteler, 4 years ago

Component: DefaultPython
Keywords: multimodule added
Milestone: 7.6.27.8.3
Version: svn-trunkgit-releasebranch78

Still broken... (test script attached):

GRASS 7.8.2dev (nc_spm_08):~ > python3 test_multimodule.py 
projection: 99 (Lambert Conformal Conic)
zone:       0
datum:      nad83
ellipsoid:  a=6378137 es=0.006694380022900787
north:      228500
south:      215000
west:       630000
east:       645000
nsres:      10
ewres:      10
rows:       1350
cols:       1500
cells:      2025000
projection: 99 (Lambert Conformal Conic)
zone:       0
datum:      nad83
ellipsoid:  a=6378137 es=0.006694380022900787
north:      228500
south:      215000
west:       630000
east:       645000
nsres:      10
ewres:      10
rows:       1350
cols:       1500
cells:      2025000
projection: 99 (Lambert Conformal Conic)
zone:       0
datum:      nad83
ellipsoid:  a=6378137 es=0.006694380022900787
north:      228500
south:      215000
west:       630000
east:       645000
nsres:      10
ewres:      10
rows:       1350
cols:       1500
cells:      2025000
 100%
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib64/python3.7/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects
 100%
 100%
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib64/python3.7/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib64/python3.7/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects

comment:4 by sbl, 2 years ago

This is fixed in master. See: https://github.com/OSGeo/grass/pull/1407 and https://lists.osgeo.org/pipermail/grass-dev/2021-May/095184.html

It has not (yet) been backported to 7.8 though due to an API change...

Can we close this or is backporting required (with GRASS 8 containing the fix on its way....)?

Note: See TracTickets for help on using tickets.