Changes between Version 44 and Version 45 of GSoC/2014/TestingFrameworkForGRASS

Timestamp:

Jul 11, 2014, 12:45:31 PM (10 years ago)

Author:

wenzeslaus

Comment:

Weekly report 8

GSoC/2014/TestingFrameworkForGRASS

@@ -596 +596 @@
 I've created a specialized functions to test output of modules such as r.info and r.univar against a reference set of values. This enables to test different properties of raster and vector maps and few other things (practically anything what is provided by some module as key-value pairs).
 
-I also added the MD5 check sum comparison of files. The functionality of Luca's g.compare.md5 is not yet available but it should be easy to add it once needed (some merge with the module should be then considered).
+I also added the MD5 check sum comparison of files. The functionality of Luca's [source:addons/grass7/general/g.compare.md5?rev=61021 g.compare.md5] is not yet available but it should be easy to add it once needed (some merge with the module should be then considered).
 
 The documentation is now updated, so although the framework is not integrated to GRASS yet, it is be now possible to write tests of GRASS modules which outputs raster and vector maps (tests of Python and ctypes function are in some limited way available from the beginning).
@@ -610 +610 @@
 Not really but it would be very helpful now to have tests of actual modules to see limitations of my current implementation.
 
-http://trac.osgeo.org/grass/browser/sandbox/wenzeslaus/gunittest/testing.rst?rev=61021
-http://trac.osgeo.org/grass/ticket/2326
-http://trac.osgeo.org/grass/browser/sandbox/wenzeslaus/gunittest/gmodules.py?rev=61003
-http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS#Week06
-
-=== Week 06 ===
+[source:sandbox/wenzeslaus/gunittest/testing.rst?rev=61021 documentation], [source:sandbox/wenzeslaus/gunittest/gmodules.py?rev=61003 gmodules Python module], [http://lists.osgeo.org/pipermail/grass-dev/2014-June/069787.html report email]
+
+=== Week 07 ===
 
 1. What did you get done this week?
@@ -632 +629 @@
 I plan to put the gutittest package to trunk. The parts needed for writing tests are more stable then the parts for invoking and report generation, so I can consider putting only the stable ones to GRASS. I need to improve command line interface, maybe introduce `make test` and finally I would like to advance in location handling (which fits to the original plan for week 8).
 
-Considering the state of report generation I can be more spacific about plan for week 10. I will improve report generation for individual test files, add some paseable outputs, and add different summaries to the main report
+Considering the state of report generation I can be more specific about plan for week 10. I will improve report generation for individual test files, add some paseable outputs, and add different summaries to the main report.
 
 3. Are you blocked on anything?
 
 I need more tests to see how the ideas about location-specific and location-independent tests work in actual tests. I will of course use the existing tests but if you can contribute some more, please do.
-Vaclav
-
-Documentation including information about locations and tests
-(additional documentation is in docstrings):
-http://trac.osgeo.org/grass/browser/sandbox/wenzeslaus/gunittest/testing.rst?rev=61144
-Wiki:
-http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS#Week07
+
+[source:sandbox/wenzeslaus/gunittest/testing.rst?rev=61144 documentation], [http://lists.osgeo.org/pipermail/grass-dev/2014-July/069923.html report email]
+
+=== Week 08 ===
+
+1. What did you get done this week?
+
+I extended the API for calling modules to allow convenient syntax, improved the naming of function and classes, added documentation, removed unnecessary code and introduced several other improvements based on the feedback of my mentor who is writing the tests for the code he maintains. I also added few more assert functions for evaluating the results. Additionally, I rewrote some existing tests to use the new testing framework and started to write test for r.slope.aspect module. Finally, I moved gunittest package from sandbox to trunk and changed the import paths and documentation accordingly.
+
+The documentation is now available in dist.../docs/html/libpython/gunittest_testing.html after running `make sphinxdoc`.
+The tests can be run for the whole GRASS using:
+
+{{{
+python -m grass.gunittest.main ...path/to/grassdata/ nc_smp_location_name nc
+
+# alternatively:
+python -m grass.gunittest.main nc_smp_location_name nc
+
+# general syntax:
+python -m grass.gunittest.main [gisdbasepath] location_name xx
+# no tests are using location identification now, so third parameter is ignored in fact
+}}}
+
+The command can be executed in any directory and it will find all tests in all subdirectories. The HTML report is stored in testreport/index.html.
+There is currently 17 successfully running tests files and 3 failing test files (each test file contains multiple tests).
+I did not advanced in testing of location switching. There is not enough tests in different locations. Anyway, main testing script switches to arbitrary location and executes each test file in unique mapset and working directory.
+
+I postponed the integration with build system. The integration basically mean that `make test` would run the main test script in the current directory in some selected location. (Demolocation looks like a good choice in this case but this will limit us to tests which are not using any data.)
+
+2. What do you plan on doing next week?
+
+The original plan for next week (week 9) is to implement missing comparison/assert methods. Currently, it is possible to compare kev-value pairs with different methods and precisions which covers most of the GRASS modules giving metadata and statistics about maps. Additional assert methods enables to compare raster maps in this way. For example, the function assertRastersNoDifference() can subtracts two raster maps and then checks their difference in a given precision. What is missing is comparison of vector and 3D raster maps and also temporal datasets.
+
+3. Are you blocked on anything?
+
+I did not solved the issue of distribution of tests for different platforms. On Unix-like systems (+- Mac OS X), I suppose that all people wanting to test will be able to compile GRASS and download all locations they are interested in. There should be some rule in makefiles to build the C test modules, so that testing framework does not have to deal with that. However, doing this on MS Windows is not likely. Perhaps some special package with GRASS source code including testsuite directories, C test modules' binaries and locations would be the way to go.
+
+[source:grass/trunk/lib/python/gunittest?rev=61238 src] [#TestingonMSWindows Testing on MS Windows section] [http://lists.osgeo.org/pipermail/grass-dev/2014-July/070000.html report email]