| | 908 | === Closing comment for ticket Missing guidelines for testing === |
| | 909 | |
| | 910 | The ticket #2105 was asking for documentation but new possible tool for testing was discussed too, so here are the points from comment:7:ticket:2105 applied to current state of testing framework. |
| | 911 | |
| | 912 | Python libraries Python: |
| | 913 | |
| | 914 | * `gunittest` |
| | 915 | * `unittest` use its assert methods through `gunittest` |
| | 916 | * `doctest` use only for documentation purposes and then test this documentation (i.e. do complex tests using `gunittest`), there is a special way how to import tests |
| | 917 | |
| | 918 | Functions in Python modules: |
| | 919 | |
| | 920 | * doctest should at least partially work, currently best to invoke directly (without testing framework) |
| | 921 | * to standardize/promote `--test` parameter for GRASS modules might be useful here |
| | 922 | * others under investigation |
| | 923 | |
| | 924 | Interface of C/C++ libraries: |
| | 925 | |
| | 926 | * test using special (test) GRASS modules, invoke these separately for benchmark and as standard GRASS modules through `gunittest` |
| | 927 | * test as Python libraries through ctypes |
| | 928 | |
| | 929 | Internal (static, private, ...) functions in C/C++ libraries: |
| | 930 | |
| | 931 | * not available (i.e., not possible to test directly/separately) |
| | 932 | |
| | 933 | Functions in C/C++ modules: |
| | 934 | |
| | 935 | * use special (test) GRASS modules as for C/C++ libraries |
| | 936 | * to standardize/promote `--test` parameter for GRASS modules might be useful here |
| | 937 | * static and private cannot be tested |
| | 938 | |
| | 939 | C/C++ modules and Python modules: |
| | 940 | |
| | 941 | * `gunittest` |
| | 942 | |
| | 943 | There are five other main areas of the testing topic: |
| | 944 | |
| | 945 | * recursive running of tests |
| | 946 | * e.g., `python -m grass.gunittest.main --location nc_spm_grass7 --location-type nc` |
| | 947 | * `make test` not implemented |
| | 948 | * reports/outputs |
| | 949 | * textual output on stdout |
| | 950 | * HTML |
| | 951 | * parsable key-value files with summaries |
| | 952 | * XML or database not implemented |
| | 953 | * automatic and online running supported using custom scripts and documented for custom server |
| | 954 | * tests currently runs every day |
| | 955 | * no online testing services explicitly supported or tested |
| | 956 | * supporting functionality especially that for comparing (maps, numbers) implemented or using `unittest` |
| | 957 | * data for testing is a complex issue but most important are random data, data included in tests and NC SPM sample location, other can be used too but this was not much tested yet |
| | 958 | |
| | 959 | |
