pygrass ignores option rules

[martinl]

PyGRASS ignores parser rules, eg.

Module('g.extension', flags='a')

ends with

Traceback (most recent call last):
...
  File "/home/landa/src/grass70_release/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 524, in __init__
    self.__call__(*args, **kargs)
  File "/home/landa/src/grass70_release/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 583, in __call__
    raise ParameterError(msg % k)
ParameterError: Required parameter <extension> not set.

Option extension is not required when -a flag is given, see

g.extension -a
List of installed extensions (modules):
v.in.ply
r.subdayprecip.design

[zarch]

ok, should be fix in r65598.

I've added a new method to check the parameters and I've also added an extra parameter (check_) to avoid the parameter validation.

[martinl]

Replying to zarch:

ok, should be fix in r65598.

I've added a new method to check the parameters and I've also added an extra parameter (check_) to avoid the parameter validation.

Thanks for quick fix!

>>> from grass.pygrass.modules import Module
>>> Module('g.extension', flags='a')
List of installed extensions (modules):
r.subdayprecip.design
r.in.proj
Module('g.extension')

Till now everything seems to be OK (except of last line where is printed Module('g.extension')).

Other test:

>>> Module('r.mask', flags='r')
ERROR: No existing MASK to remove
List of installed extensions (modules):
r.subdayprecip.design
r.in.proj
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 526, in __init__
    self.__call__(*args, **kargs)
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 583, in __call__
    return self.run()
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/exceptions/__init__.py", line 68, in __init__
    msg = _("Module run %s %s ended with error") % (module, code)
TypeError: unsupported operand type(s) for %: 'Module' and 'tuple'

It also prints output of previous command (g.extensions) and ends with error.

[zarch]

Replying to martinl:

Replying to zarch:

>>> from grass.pygrass.modules import Module
>>> Module('g.extension', flags='a')
List of installed extensions (modules):
r.subdayprecip.design
r.in.proj
Module('g.extension')

Till now everything seems to be OK (except of last line where is printed Module('g.extension')).

The current behavior of the Module return the instance itself, but since in the standard python you cannot see the difference, but it is much clear on an ipython shell:

In [1]: from grass.pygrass.modules import Module

In [2]: Module('g.extension', flags='a')
List of installed extensions (modules):
r.skyview
r.sun.hourly
r.green
Out[2]: Module('g.extension')

Other test:

>>> Module('r.mask', flags='r')
ERROR: No existing MASK to remove
List of installed extensions (modules):
r.subdayprecip.design
r.in.proj
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 526, in __init__
    self.__call__(*args, **kargs)
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 583, in __call__
    return self.run()
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/exceptions/__init__.py", line 68, in __init__
    msg = _("Module run %s %s ended with error") % (module, code)
TypeError: unsupported operand type(s) for %: 'Module' and 'tuple'

It also prints output of previous command (g.extensions) and ends with error.

Actually I'm not able to reproduce it, it seems to work properly:

In [1]: from grass.pygrass.modules import Module

In [2]: Module('r.mask', raster='elevation')
All subsequent raster operations will be limited to the MASK area. Removing
or renaming raster map named 'MASK' will restore raster operations to
normal.
Out[2]: Module('r.mask')

In [3]: Module('r.mask', flags='r')
Raster MASK removed
Out[3]: Module('r.mask')

In [4]: Module('r.mask', flags='r')
ERROR: No existing MASK to remove
---------------------------------------------------------------------------
CalledModuleError                         Traceback (most recent call last)
<ipython-input-4-d9799e394d0a> in <module>()
----> 1 Module('r.mask', flags='r')

/home/pietro/docdat/src/gis/grass71/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.pyc in __init__(self, cmd, *args, **kargs)
    524 
    525         if args or kargs:
--> 526             self.__call__(*args, **kargs)
    527         self.__call__.__func__.__doc__ = self.__doc__
    528 

/home/pietro/docdat/src/gis/grass71/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.pyc in __call__(self, *args, **kargs)
    581             if self.check_:
    582                 self.check()
--> 583             return self.run()
    584         return self
    585 

/home/pietro/docdat/src/gis/grass71/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.pyc in run(self)
    711                 raise CalledModuleError(returncode=self.popen.returncode,
    712                                         code=self.get_bash(),
--> 713                                         module=self.name, errors=stderr)
    714         return self
    715 

CalledModuleError: Module run r.mask r.mask maskcats=* layer=1 -r ended with error
Process ended with non-zero return code 1. See errors in the (error) output.

[martinl]

Replying to zarch:

The current behavior of the Module return the instance itself, but since in the standard python you cannot see the difference, but it is much clear on an ipython shell:

Thanks for explanation.

Actually I'm not able to reproduce it, it seems to work properly:

> In [1]: from grass.pygrass.modules import Module

I did test on other computer where I could reproduce it. Please try to launch the second command after the first command in one session without importing pygrass package twice.

>>> from grass.pygrass.modules import Module
>>> Module('g.extension', flags='a')
List of installed extensions (modules):
v.in.ply
r.subdayprecip.design
Module('g.extension')
>>> Module('r.mask', flags='r')     
ERROR: No existing MASK to remove
List of installed extensions (modules):
v.in.ply
r.subdayprecip.design
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 526, in __init__
    self.__call__(*args, **kargs)
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 583, in __call__
    return self.run()
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/exceptions/__init__.py", line 68, in __init__
    msg = _("Module run %s %s ended with error") % (module, code)
TypeError: unsupported operand type(s) for %: 'Module' and 'tuple'
>>> 

[martinl]

I can confirm this bug also in relbr70. It's relevant only when using Python interactive prompt. Python scripts work as expected:

GRASS 7.1.svn (s-42):~/smetiste > cat pytest.py 
#!/usr/bin/env python

from grass.pygrass.modules import Module
Module('g.extension', flags='a')
Module('r.mask', flags='r')
GRASS 7.1.svn (s-42):~/smetiste > ./pytest.py 
List of installed extensions (modules):
r.subdayprecip.design
r.in.proj
ERROR: No existing MASK to remove
Traceback (most recent call last):
  File "./pytest.py", line 5, in <module>
    Module('r.mask', flags='r')
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 526, in __init__
    self.__call__(*args, **kargs)
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 583, in __call__
    return self.run()
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)
grass.exceptions.CalledModuleError: Module run r.mask r.mask maskcats=* layer=1 -r ended with error
Process ended with non-zero return code 1. See errors in the (error) output.

vs.

GRASS 7.1.svn (s-42):~/smetiste > python
Python 2.7.10 (default, Jul  1 2015, 10:54:53) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from grass.pygrass.modules import Module
>>> Module('g.extension', flags='a')
List of installed extensions (modules):
r.subdayprecip.design
r.in.proj
Module('g.extension')
>>> Module('r.mask', flags='r')
ERROR: No existing MASK to remove
List of installed extensions (modules):
r.subdayprecip.design
r.in.proj
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 526, in __init__
    self.__call__(*args, **kargs)
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 583, in __call__
    return self.run()
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/exceptions/__init__.py", line 68, in __init__
    msg = _("Module run %s %s ended with error") % (module, code)
TypeError: unsupported operand type(s) for %: 'Module' and 'tuple'

[martinl]

The problem also disappears when storing Module object to a variable, see

>>> from grass.pygrass.modules import Module
>>> m1 = Module('g.extension', flags='a')
List of installed extensions (modules):
r.subdayprecip.design
r.in.proj
>>> m2 = Module('r.mask', flags='r')
ERROR: No existing MASK to remove
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 526, in __init__
    self.__call__(*args, **kargs)
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 583, in __call__
    return self.run()
  File "/opt/src/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)
grass.exceptions.CalledModuleError: Module run r.mask r.mask maskcats=* layer=1 -r ended with error
Process ended with non-zero return code 1. See errors in the (error) output.

[martinl]

Decreasing priority since it's related only to interactive Python prompt.

[zarch]

Replying to martinl:

I did test on other computer where I could reproduce it. Please try to launch the second command after the first command in one session without importing pygrass package twice.

You are right I have the same issue I'm very confused, I don't understand what is going on... I don't have this problem if:

• I use the ipython interpreter
• using python3
• using the standard python interpreter but executing the code from an external file
• running from the command line python -c "from grass.pygrass.modules import Module; Module('g.extension', flags='a'); Module('r.mask', flags='r')"
• or using the standard interpreter If I save the instance of the module.

$ python
Python 2.7.10 (default, May 26 2015, 04:16:29) 
[GCC 5.1.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from grass.pygrass.modules import Module
>>> gext = Module('g.extension', flags='a')
List of installed extensions (modules):
r.skyview
r.sun.hourly
r.green
>>> Module('r.mask', flags='r')
ERROR: No existing MASK to remove
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pietro/docdat/src/gis/grass71/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 526, in __init__
    self.__call__(*args, **kargs)
  File "/home/pietro/docdat/src/gis/grass71/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 583, in __call__
    return self.run()
  File "/home/pietro/docdat/src/gis/grass71/dist.x86_64-unknown-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)
grass.exceptions.CalledModuleError: Module run r.mask r.mask maskcats=* layer=1 -r ended with error
Process ended with non-zero return code 1. See errors in the (error) output.

What could be the problem?

[martinl]

Replying to zarch:

ok, should be fix in r65598.

I've added a new method to check the parameters and I've also added an extra parameter (check_) to avoid the parameter validation.

Please could you backport it to relbr70 (I am not sure if there are more related commits)?

[martinl]

Replying to martinl:

Replying to zarch:

ok, should be fix in r65598.

I took liberty to backport it to relbr70 in r65605. Since original problem is solved I am closing this ticket. I also created the new ticket #2709 for related problem.