Python testsuite failing with UnicodeDecodeError

[marisn]

Unfortunately it is not possible to run tests as test suite is failing itself (has anyone written tests for test suite?):

ERROR: test_seed_required (__main__.TestRandFunction)
Test that seed is required when rand() is used
----------------------------------------------------------------------
Traceback (most recent call last):
  File "raster/r.mapcalc/testsuite/test_r_mapcalc.py", line 99, in test_seed_required
    self.assertModuleFail('r.mapcalc', expression='rand_x = rand(1, 200)')
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py", line 1180, in assertModuleFail
    module.run()
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/exceptions/__init__.py", line 71, in __init__
    msg += _(". See the following errors:\n%s") % errors
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 29: ordinal not in range(128)

The largest question is - should there be strings for translation at all? I think not as most of low level errors and debug information is not marked for translation. Tests should be run by developers not end users thus I don't see a point in translating such strings.

[mlennert]

Replying to marisn:

Unfortunately it is not possible to run tests as test suite is failing itself (has anyone written tests for test suite?):

ERROR: test_seed_required (__main__.TestRandFunction)
Test that seed is required when rand() is used
----------------------------------------------------------------------
Traceback (most recent call last):
  File "raster/r.mapcalc/testsuite/test_r_mapcalc.py", line 99, in test_seed_required
    self.assertModuleFail('r.mapcalc', expression='rand_x = rand(1, 200)')
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py", line 1180, in assertModuleFail
    module.run()
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)
  File "/home/maris/soft/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/exceptions/__init__.py", line 71, in __init__
    msg += _(". See the following errors:\n%s") % errors
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 29: ordinal not in range(128)

The largest question is - should there be strings for translation at all? I think not as most of low level errors and debug information is not marked for translation. Tests should be run by developers not end users thus I don't see a point in translating such strings.

Generally +1, but maybe a few tests specifically on encoding might help finding encoding errors automatically. Although most encoding issues are GUI-related and I don't know if it is possible to test that ?

[wenzeslaus]

Replying to marisn:

  File ".../pygrass/modules/interface/module.py", line 713, in run
  File ".../exceptions/__init__.py", line 71, in __init__

The largest question is - should there be strings for translation at all? I think not as most of low level errors and debug information is not marked for translation. Tests should be run by developers not end users thus I don't see a point in translating such strings.

The issue is that gunittest package currently depends on PyGRASS modules and few other things (grass.exceptions in this case). In general, there is no reason why the the dependency should be there if there is an alternative implementation (and we are fine with some code duplication).

[wenzeslaus]

Replying to mlennert:

...maybe a few tests specifically on encoding might help finding encoding errors automatically. Although most encoding issues are GUI-related and I don't know if it is possible to test that ?

Test suites can call whatever they want. Try to use translations or call some parts of GUI code - of course only the one which is actually not doing any GUI, there are some examples already. When calling module, whatever is in the module, e.g. translated error messages, is applied as usually (no special settings or code in gunittests to handle that).

[wenzeslaus]

Replying to marisn:

Unfortunately it is not possible to run tests as test suite is failing itself...

    module.run()
  File ".../grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)

Since this comes from grass.pygrass.modules, can you try the Module class separately?

[neteler]

Milestone renamed

[neteler]

Ticket retargeted after milestone closed

[marisn]

Replying to wenzeslaus:

Replying to marisn:

Unfortunately it is not possible to run tests as test suite is failing itself...

    module.run()
  File ".../grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)

Since this comes from grass.pygrass.modules, can you try the Module class separately?

Can you provide code for testing? I was just trying to run some of existing tests to see how they behave. Unfortunately it currently blocks me from writing any tests based on existing examples as none of tests works for me.

At the moment this bug is blocking developing tests for v.profile (a requirement to move from addons to main modules) and also providing a tested test for #3296.

[wenzeslaus]

Replying to marisn:

Replying to wenzeslaus:

Replying to marisn:

Unfortunately it is not possible to run tests as test suite is failing itself...

    module.run()
  File ".../grass/pygrass/modules/interface/module.py", line 713, in run
    module=self.name, errors=stderr)

Since this comes from grass.pygrass.modules, can you try the Module class separately?

Can you provide code for testing?

Please use any PyGRASS documentation to execute a module using the Module class (open an a separate ticket if you can't find it). It should be something like (pseudo-code based on the above):

module = Module('r.mapcalc', expression='rand_x = rand(1, 200)')
module.run()
# An error should occur because we use ''rand()'' without a seed info.

[marisn]

It seems to work as expected (and tests are still failing):

python -c 'from grass.pygrass.modules import Module; module = Module("r.mapcalc", expression="rand_x = rand(1, 200)")'
KĻŪDA: Pseudo-random number generator not seeded
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 591, in __init__
    self.__call__(*args, **kargs)
  File "dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 648, in __call__
    return self.run()
  File "dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 769, in run
    self.wait()
  File "dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 790, in wait
    module=self.name, errors=stderr)
grass.exceptions.CalledModuleError: Module run r.mapcalc r.mapcalc expression=rand_x = rand(1, 200) ended with error
Process ended with non-zero return code 1. See errors in the (error) output.

[marisn]

Also reported as #3411

[mlennert]

Sorry, had forgotten about this ticket. Thanks Maris for closing the other.

Coming back to the debate concerning localization of the testing suite. I don't think that the output of the testing suite has to be localized, but I actually notice that a series of tests fail because the messages from the tested modules, but others because the testing is based on C-locale output of the module (e.g. g.remove).

I think it would be great to be able to run the entire test suite in any locale. This would allow more systematic testing of locale related issues in modules. I don't think that the outputs of the testsuite have to be localized, but errors arise because of the handling of module error messages by the testsuite:

python test_r_mapcalc.py 


.

.

.

.



.

.

.

.

.E
======================================================================
ERROR: test_seed_required (__main__.TestRandFunction)
Test that seed is required when rand() is used
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_r_mapcalc.py", line 99, in test_seed_required
    self.assertModuleFail('r.mapcalc', expression='rand_x = rand(1, 200)')
  File "/data/home/mlennert/SRC/GRASS/grass-7.2.2RC2/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py", line 1180, in assertModuleFail
    module.run()
  File "/data/home/mlennert/SRC/GRASS/grass-7.2.2RC2/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py", line 716, in run
    module=self.name, errors=stderr)
  File "/data/home/mlennert/SRC/GRASS/grass-7.2.2RC2/dist.x86_64-pc-linux-gnu/etc/python/grass/exceptions/__init__.py", line 71, in __init__
    msg += _(". See the following errors:\n%s") % errors
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 37: ordinal not in range(128)

----------------------------------------------------------------------
Ran 10 tests in 0.983s
FAILED (errors=1)

Running the same test with LANGUAGE=C gives me:

LANGUAGE=C;python test_r_mapcalc.py 


.

.

.

.



.

.

.

.

.
ERROR: Pseudo-random number generator not seeded

.
----------------------------------------------------------------------
Ran 10 tests in 1.014s
OK

IIUC, the difference comes from this:

LANGUAGE=fr_BE;r.mapcalc expression='rand_x = rand(1, 200)'
ERREUR :Générateur de nombres pseudo-aléatoires sans graine
GRASS 7.2.2RC2 (nc_spm_08):/data/home/mlennert/SRC/GRASS/grass-7.2.2RC2/raster/r.mapcalc/testsuite > LANGUAGE=C;r.mapcalc expression='rand_x = rand(1, 200)'
ERROR: Pseudo-random number generator not seeded

[mlennert]

Another illustration running the entire testsuite in the 7.2.2RC2 code tree:

LANGUAGE=fr_BE;python -m grass.gunittest.main --location nc_spm_08 --location-type nc
...
Executed 156 test files in 0:16:00.337352.
From them 81 files (52%) were successful and 75 files (48%) failed.

and

LANGUAGE=C;python -m grass.gunittest.main --location nc_spm_08 --location-type nc
Executed 156 test files in 0:22:05.879940.
From them 131 files (84%) were successful and 25 files (16%) failed.

(of those 16% some failed because I used the full nc_spm data set, not the basic data set, and tests seem to be for the latter. I couldn't find any documentation on that)

[neteler]

Ticket retargeted after milestone closed

[martinl]

Ticket retargeted after milestone closed

[neteler]

Replying to mlennert:

(of those 16% some failed because I used the full nc_spm data set, not the basic data set, and tests seem to be for the latter. I couldn't find any documentation on that)

Indeed, the dataset-to-be-used is lacking from here:

https://grass.osgeo.org/grass77/manuals/libpython/gunittest_testing.html

[marisn]

This particular case seems to be fixed. Could not reproduce any more with 7.7.