Opened 5 years ago

Closed 19 months ago

#2875 closed defect (fixed)

Python testsuite failing with UnicodeDecodeError

Reported by: marisn Owned by: grass-dev@…
Priority: normal Milestone: 7.2.4
Component: Tests Version: svn-trunk
Keywords: Cc:
CPU: Unspecified Platform: Linux

Description

Unfortunately it is not possible to run tests as test suite is failing itself (has anyone written tests for test suite?):

ERROR: test_seed_required (__main__.TestRandFunction)
Test that seed is required when rand() is used
----------------------------------------------------------------------
Traceback (most recent call last):
  File "raster/r.mapcalc/testsuite/test_r_mapcalc.py", line 99, in test_seed_required
    self.assertModuleFail('r.mapcalc', expression='rand_x = rand(1, 200)')
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py", line 1180, in assertModuleFail
    module.run()
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/exceptions/__init__.py", line 71, in __init__
    msg += _(". See the following errors:\n%s") % errors
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 29: ordinal not in range(128)

The largest question is - should there be strings for translation at all? I think not as most of low level errors and debug information is not marked for translation. Tests should be run by developers not end users thus I don't see a point in translating such strings.

Change History (18)

comment:1 in reply to:  description ; Changed 5 years ago by mlennert

Replying to marisn:

Unfortunately it is not possible to run tests as test suite is failing itself (has anyone written tests for test suite?):

ERROR: test_seed_required (__main__.TestRandFunction)
Test that seed is required when rand() is used
----------------------------------------------------------------------
Traceback (most recent call last):
  File "raster/r.mapcalc/testsuite/test_r_mapcalc.py", line 99, in test_seed_required
    self.assertModuleFail('r.mapcalc', expression='rand_x = rand(1, 200)')
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py", line 1180, in assertModuleFail
    module.run()
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/exceptions/__init__.py", line 71, in __init__
    msg += _(". See the following errors:\n%s") % errors
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 29: ordinal not in range(128)

The largest question is - should there be strings for translation at all? I think not as most of low level errors and debug information is not marked for translation. Tests should be run by developers not end users thus I don't see a point in translating such strings.

Generally +1, but maybe a few tests specifically on encoding might help finding encoding errors automatically. Although most encoding issues are GUI-related and I don't know if it is possible to test that ?

comment:2 in reply to:  description Changed 5 years ago by wenzeslaus

Replying to marisn:

  File ".../pygrass/modules/interface/module.py", line 713, in run
  File ".../exceptions/__init__.py", line 71, in __init__

The largest question is - should there be strings for translation at all? I think not as most of low level errors and debug information is not marked for translation. Tests should be run by developers not end users thus I don't see a point in translating such strings.

The issue is that gunittest package currently depends on PyGRASS modules and few other things (grass.exceptions in this case). In general, there is no reason why the the dependency should be there if there is an alternative implementation (and we are fine with some code duplication).

comment:3 in reply to:  1 Changed 5 years ago by wenzeslaus

Replying to mlennert:

...maybe a few tests specifically on encoding might help finding encoding errors automatically. Although most encoding issues are GUI-related and I don't know if it is possible to test that ?

Test suites can call whatever they want. Try to use translations or call some parts of GUI code - of course only the one which is actually not doing any GUI, there are some examples already. When calling module, whatever is in the module, e.g. translated error messages, is applied as usually (no special settings or code in gunittests to handle that).

comment:4 in reply to:  description ; Changed 5 years ago by wenzeslaus

Replying to marisn:

Unfortunately it is not possible to run tests as test suite is failing itself...

    module.run()
  File ".../grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)

Since this comes from grass.pygrass.modules, can you try the Module class separately?

comment:5 Changed 5 years ago by neteler

Milestone: 7.1.07.2.0

Milestone renamed

comment:6 Changed 4 years ago by neteler

Milestone: 7.2.07.2.1

Ticket retargeted after milestone closed

comment:7 in reply to:  4 ; Changed 4 years ago by marisn

Replying to wenzeslaus:

Replying to marisn:

Unfortunately it is not possible to run tests as test suite is failing itself...

    module.run()
  File ".../grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)

Since this comes from grass.pygrass.modules, can you try the Module class separately?

Can you provide code for testing? I was just trying to run some of existing tests to see how they behave. Unfortunately it currently blocks me from writing any tests based on existing examples as none of tests works for me.

At the moment this bug is blocking developing tests for v.profile (a requirement to move from addons to main modules) and also providing a tested test for #3296.

comment:8 in reply to:  7 Changed 4 years ago by wenzeslaus

Replying to marisn:

Replying to wenzeslaus:

Replying to marisn:

Unfortunately it is not possible to run tests as test suite is failing itself...

    module.run()
  File ".../grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)

Since this comes from grass.pygrass.modules, can you try the Module class separately?

Can you provide code for testing?

Please use any PyGRASS documentation to execute a module using the Module class (open an a separate ticket if you can't find it). It should be something like (pseudo-code based on the above):

module = Module('r.mapcalc', expression='rand_x = rand(1, 200)')
module.run()
# An error should occur because we use ''rand()'' without a seed info.

comment:9 Changed 4 years ago by marisn

It seems to work as expected (and tests are still failing):

python -c 'from grass.pygrass.modules import Module; module = Module("r.mapcalc", expression="rand_x = rand(1, 200)")'
KĻŪDA: Pseudo-random number generator not seeded
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 591, in __init__
    self.__call__(*args, **kargs)
  File "dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 648, in __call__
    return self.run()
  File "dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 769, in run
    self.wait()
  File "dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 790, in wait
    module=self.name, errors=stderr)
grass.exceptions.CalledModuleError: Module run r.mapcalc r.mapcalc expression=rand_x = rand(1, 200) ended with error
Process ended with non-zero return code 1. See errors in the (error) output.

comment:10 Changed 4 years ago by martinl

Milestone: 7.2.17.2.2

comment:11 Changed 3 years ago by marisn

Also reported as #3411

comment:12 Changed 3 years ago by mlennert

Sorry, had forgotten about this ticket. Thanks Maris for closing the other.

Coming back to the debate concerning localization of the testing suite. I don't think that the output of the testing suite has to be localized, but I actually notice that a series of tests fail because the messages from the tested modules, but others because the testing is based on C-locale output of the module (e.g. g.remove).

I think it would be great to be able to run the entire test suite in any locale. This would allow more systematic testing of locale related issues in modules. I don't think that the outputs of the testsuite have to be localized, but errors arise because of the handling of module error messages by the testsuite:

python test_r_mapcalc.py 


.

.

.

.



.

.

.

.

.E
======================================================================
ERROR: test_seed_required (__main__.TestRandFunction)
Test that seed is required when rand() is used
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_r_mapcalc.py", line 99, in test_seed_required
    self.assertModuleFail('r.mapcalc', expression='rand_x = rand(1, 200)')
  File "/data/home/mlennert/SRC/GRASS/grass-7.2.2RC2/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py", line 1180, in assertModuleFail
    module.run()
  File "/data/home/mlennert/SRC/GRASS/grass-7.2.2RC2/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 716, in run
    module=self.name, errors=stderr)
  File "/data/home/mlennert/SRC/GRASS/grass-7.2.2RC2/dist.x86_64-pc-linux-gnu/etc/python/grass/exceptions/__init__.py", line 71, in __init__
    msg += _(". See the following errors:\n%s") % errors
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 37: ordinal not in range(128)

----------------------------------------------------------------------
Ran 10 tests in 0.983s
FAILED (errors=1)

Running the same test with LANGUAGE=C gives me:

LANGUAGE=C;python test_r_mapcalc.py 


.

.

.

.



.

.

.

.

.
ERROR: Pseudo-random number generator not seeded

.
----------------------------------------------------------------------
Ran 10 tests in 1.014s
OK

IIUC, the difference comes from this:

LANGUAGE=fr_BE;r.mapcalc expression='rand_x = rand(1, 200)'
ERREUR :Générateur de nombres pseudo-aléatoires sans graine
GRASS 7.2.2RC2 (nc_spm_08):/data/home/mlennert/SRC/GRASS/grass-7.2.2RC2/raster/r.mapcalc/testsuite > LANGUAGE=C;r.mapcalc expression='rand_x = rand(1, 200)'
ERROR: Pseudo-random number generator not seeded

comment:13 Changed 3 years ago by mlennert

Another illustration running the entire testsuite in the 7.2.2RC2 code tree:

LANGUAGE=fr_BE;python -m grass.gunittest.main --location nc_spm_08 --location-type nc
...
Executed 156 test files in 0:16:00.337352.
From them 81 files (52%) were successful and 75 files (48%) failed.

and

LANGUAGE=C;python -m grass.gunittest.main --location nc_spm_08 --location-type nc
Executed 156 test files in 0:22:05.879940.
From them 131 files (84%) were successful and 25 files (16%) failed.

(of those 16% some failed because I used the full nc_spm data set, not the basic data set, and tests seem to be for the latter. I couldn't find any documentation on that)

comment:14 Changed 3 years ago by neteler

Milestone: 7.2.27.2.3

Ticket retargeted after milestone closed

comment:15 Changed 3 years ago by martinl

Milestone: 7.2.3

Ticket retargeted after milestone closed

comment:16 Changed 3 years ago by martinl

Milestone: 7.2.4

comment:17 in reply to:  13 Changed 23 months ago by neteler

Replying to mlennert:

(of those 16% some failed because I used the full nc_spm data set, not the basic data set, and tests seem to be for the latter. I couldn't find any documentation on that)

Indeed, the dataset-to-be-used is lacking from here:

https://grass.osgeo.org/grass77/manuals/libpython/gunittest_testing.html

comment:18 Changed 19 months ago by marisn

Resolution: fixed
Status: newclosed

This particular case seems to be fixed. Could not reproduce any more with 7.7.

Note: See TracTickets for help on using tickets.