GSoC 2018 Improve GRASS GIS integration in QGIS 3
| Title: | Improve GRASS GIS integration in QGIS 3 |
| Student Name: | Radek Novotný, Czech Technical University in Prague |
| Organization: | OSGeo - Open Source Geospatial Foundation |
| Mentor Name: | Vaclav Petras, Marin Landa |
| GitHub Repository: | view repository |
| GSoC proposal: | view proposal |
Abstract
Currently, GRASS integration in QGIS is hard to maintain. There are two QGIS plugins - Processing and GRASS Plugin, which differs each other. This proposal is focused on Processing plugin. Processing plugin is at this moment maintained by QGIS developers in core Git repository. This situation has been discussed in the QGIS and GRASS GIS mailing lists several times. GUI dialogues of the Processing plugin are generated from manually maintained text UI description files.
Goal
My goal is to design a Python library (and possibly also a GRASS module based on this library) which allows creating this text UI files automatically from generated GRASS XML description. This library also has to simplify and divide parameters for QGIS. This approach would be similar to the function of SAGA (or OTB) Processing plugin integration QGIS.
After this addition will be possible to generate text UI files for new GRASS modules (added to the core or even for add-ons) in an automated way. These generated UI files can be part of Git repository, but in future could be generated automatically during QGIS compilation, when a user installs GRASS Addon module using Processing plugin, or on the fly when the GUI is started.
Timeline
Time Period | Milestones | |
|---|---|---|
Tasks | Status | |
April 23 - May 14 |
|
Ok |
May 14 - May 20 |
| Done |
May 21 - May 27 |
| Done |
May 28 - June 3 |
| Postponed |
June 4 - June 10 |
| In progress |
July 11 - July 15 |
| |
June 11 - June 17 |
| In progress |
June 18 - June 24 |
| In progress |
June 25 - July 1 |
| |
July 2 - July 8 |
| |
July 9 - July 13 |
| |
July 9 - July 15 |
| |
July 16 - July 22 |
| |
July 23 - July 29 |
| |
July 30 - August 5 |
| |
August 6 - August 14 |
| |
Suggested timeline for final period of GSoC
July 16 - July 22 |
| |
July 23 - July 29 |
| |
July 30 - August 5 |
| |
August 6 - August 14 |
|
Evaluation 2 Status
Screenshots of current version of GUI from created description from parsing library.
Bonding period report
Introduce myself in dev list, get in contact with my mentors and discuss the design of functions
After being accepted as a student for GSoC 2018, I introduced my self in grass-dev lists on 24th of April (1). Also, I got in contact with my mentors - Marin Landa and Vaclav Petras. I discussed with them how to set-up my dev environment and ask how the functions should be designed.
Prepare the wiki page about the project
I created my project wiki page (2) and added the link to the GSoC 2018 Accepted proposals page (3).
The wiki page includes - General information about the project (title, mentors, links to the proposal and GitHub repository, etc.), a brief description of the project, goal and timeline of tasks and deliverables.
I will keep my wiki page constantly up to date and I’ll add weekly reports following the instructions in the GSoC Recommendations for Student page (4).
Set up the GitHub repository of the project
I chose GitHub (5) as a public repository for the development of this project. I added the link to the GSoC 2018 Accepted proposals page (3), to my wiki page (2) and I shared it with my mentors. My repository is licensed under the GNU General Public License v3.0, according to the licence of GRASS GIS.
Set up a developer environment
I checked and updated my developer environment so as to be ready to start coding after the bonding period.
(1) https://lists.osgeo.org/pipermail/grass-dev/2018-April/088262.html
(2) https://trac.osgeo.org/grass/wiki/GSoC/2018/IntegrationInQGIS3
(3) https://wiki.osgeo.org/wiki/Google_Summer_of_Code_2018_Accepted
(4) https://wiki.osgeo.org/wiki/Google_Summer_of_Code_Recommendations_for_Students
(5) https://github.com/radeknovotny94/GRASSIntegrationInQGIS3
Weekly reports
Week 1
1) What did I complete this week?
I started work on implementing an initial version of Python library for generating UI files used by Processing plugin based on GRASS modules XML description. Unfortunately, I was not able to finish this task in time, because of I stuck in an automated generation of XML description from GRASS (1). So I have not enough sources for implementation and test of the parser (2).
2) What am I going to achieve for next week?
I want to finish work from the Week 1 and also complete the goal of week 2 - Integration of generated description UI files from initial Python library to Processing plugin in order to generate GUI dialogue for GRASS modules.
3) Is there any blocking issue?
I am blocked with generating XML descriptions from GRASS. (1)
(1) https://github.com/radeknovotny94/GRASS_Parse_to_QGIS_UI/blob/master/generate_batch_for_xml.py
(2) https://github.com/radeknovotny94/GRASS_Parse_to_QGIS_UI/blob/master/GRASSDescribtionParser.py
Week 2
What did you get done this week?
- Implementation of an initial version of the library for parsing description files for QGIS Processing plugin. I solved the problem with creating XML files from week 1. (1)
- Create an independent version of Processing plugin for development. (2)
- Try to create GUI Integration from generated description UI files from initial Python library to Processing plugin.
What do you plan on doing next week?
- Final Python library for parsing descriptions.
- Deep integration of UI descriptions to GUI Processing plugin.
- Split GRASS module parameters in Processing plugin to basic and advanced tabs.
Are you blocked on anything?
No, at the moment I’m not blocked.
(1) https://github.com/radeknovotny94/GRASS_Parse_to_QGIS_UI
(2) https://github.com/radeknovotny94/GRASSIntegrationInQGIS3
Week 3
What did you get done this week?
- I was working on the improvement of the library for parsing description files for QGIS Processing plugin. (1)
- I test creating of GUI from descriptions.
What do you plan on doing next week?
- Final Python library for parsing descriptions.
- Split GRASS module parameters in Processing plugin to basic and advanced tabs.
Are you blocked on anything?
- I have some questions about different ways in QGIS and GRASS, I will discuss them with my mentors.
(1) https://github.com/radeknovotny94/GRASS_Parse_to_QGIS_UI
Week 4
What did you get done this week?
- I finalized the library for parsing description files for QGIS Processing plugin. But of course I suppose, there will occur some bugs, after testing modules in QGIS Processing plugin. (1)
- Add 'Controversy decision' in building parser to wiki. (2)
What do you plan on doing next week?
- Split GRASS module parameters in Processing plugin to basic and advanced tabs - it should be required and optional tab.
- Check GRASS X QGIS parameter descriptions in order to take care of differences in terminology.
Are you blocked on anything?
- No, at the moment I’m not blocked.
(1) https://github.com/radeknovotny94/GRASS_Parse_to_QGIS_UI
(2) https://trac.osgeo.org/grass/wiki/GSoC/2018/IntegrationInQGIS3
Week 5
What did you get done this week?
Unfortunately, I did not have a lot of time for the project this week, because I have exams at my university.
- I improved the library for parsing description files for QGIS Processing plugin. (1)
- Add all 'Controversy decision' in project and summary from hangouts to wiki. (2)
What do you plan on doing next week?
- Split GRASS module parameters in Processing plugin to tabs - it should be on GUI Tabs.
- Find all differences in terminology of QGIS and GRASS GIS.
Are you blocked on anything?
- No, at the moment I’m not blocked.
(1) https://github.com/radeknovotny94/GRASS_Parse_to_QGIS_UI
(2) https://trac.osgeo.org/grass/wiki/GSoC/2018/IntegrationInQGIS3
Week 6
What did you get done this week?
I had last week of my exams at my university, hence a progress in the project is not big.
- I continued with bug fixing in parser library (1)
- Split GRASS module parameters in Processing plugin to tabs was postponed. See (2)
What do you plan on doing next week?
- Fix bugs in parsing library
- Research how modules and parameters are divided at this moment and also check terminology.
Are you blocked on anything?
- No, at the moment I’m not blocked.
For more see wiki (3)
(1) https://github.com/radeknovotny94/GRASS_Parse_to_QGIS_UI
(2) http://osgeo-org.1560.x6.nabble.com/Re-Improve-GRASS-GIS-integration-in-QGIS-3-Week-3-Report-tp5368107p5368128.html
(3) https://trac.osgeo.org/grass/wiki/GSoC/2018/IntegrationInQGIS3
Week 7
What did you get done this week?
- I continued with bug fixing in parser library (1)
- I did research on how parameters are divided at this moment and also check terminology.
What do you plan on doing next week?
- Fix bugs in parsing library
- Resolve terminology differences
Are you blocked on anything?
- No, at the moment I’m not blocked.
For more see wiki (2)
(1) https://github.com/radeknovotny94/GRASS_Parse_to_QGIS_UI/commit/0d9fc2f6d7d255e9e5d38a0b338c44b06942f852
(2) https://trac.osgeo.org/grass/wiki/GSoC/2018/IntegrationInQGIS3
Week 8
What did you get done this week?
- Unfortunately, I was all week on camp with my sports team, there was no electricity and the internet. So I was not able to work on the project, I hope I will be able to catch up with the delay next week.
What do you plan on doing next week?
- Fix bugs in parsing library
- Resolve terminology differences
Are you blocked on anything?
- No, at the moment I’m not blocked.
Notes from discussions
June 4: Hangout - Vaclav Petras, Radek Novotny
How to parse questionable parameters?
Raster 3D
If 'prompt' is 'raster_3d'
- if this parameter is required - delete description file for QGIS Processing, because Processing does not have 3D Raster class. Could be resolve later.
- if it is optional - pass this parameter
- in both cases is give information about the parameter to log
Examples: Not required parameter: r.colors Not recognized >>> raster_3d
Required parameter: r.to.rast3 Need of 3D raster for module >>> r.to.rast3
Group input/output
If 'prompt' is 'group':
- only in module 'i.group' is parameter 'group' output
- in all other modules should be 'group' input
-if it is in 'r.' module, it is treated like raster multilayer -if it is in 'v.' module, it is treated like vector multilayer
What is treating as the string in QGIS Processing:
Separators - 'separator' and all names which have not equivalent class in Processing plugin - 'color', 'cats', 'dbname', 'dbtable'.
GRASS layer, sublayer:
GRASS GIS layer and sublayer is passed because it is no equivalent in QGIS.
June 11: Hangout - Vaclav Petras, Martin Landa, Radek Novotny
How solve splitting modules in QGIS
r.slope.aspect in GRASS GIS vs. r.slope, r.aspect, ... in QGIS Processing plugin
Reasons for splitting:
- Users are used to modules/tools doing one single thing (slope only versus many other things)
- Too many options
Conclusion: More intuitive distribution of parameters in GRASS GIS (more in Future to do)
What is the best way how to divide parameters of modules in Processing plugin QGIS?
Discussed possibilities:
- Required x Optional - This version was declined, because of a lot of important parameters are optional in GRASS modules.
- Frequently used x Rarely used - This system was declined for now, because it is not clear what parameter is rarely or often used. But it could be a good point to think for future because this distribution has to be first done in GRASS GIS.
- GUI Tabs - We decided for distribution according to the GUI sections in GRASS. It should be the least confusing system for users because it will be same in QGIS and GRASS. Also for future changes and maintenance, it appears like a good solution.
How solved differences in GRASS and QGIS terminology.
The biggest problem is that descriptions of parameters are different between GRASS and QGIS now and simultaneously parameter names are not shown in QGIS Processing plugin. So it is very difficult for a user to orientate in manuals.
There are following variants:
- what is possible to change in GRASS, so change it. E.g. 'raster map' in GRASS x 'raster layer' in QGIS - could be solved as 'raster input' or 'raster output' without the word map or layer.
- what is not possible change in GRASS, then add a new tag with OGC (or QGIS) friendly description. This description could be shown in manuals too.
Because that's a big deal and have to be thoroughly discussed in GRASS Mailing list. I will try to find all differences in QGIS and GRASS terminology and report them to ML as a source for a final decision.
Future to do:
- add keywords to Processing Plugin - for improvement of searching in modules
- add similar keywords to ArcGIS tools to modules - for better orientation for ArcGIS users
- show parameters in Processing Plugin - for better orientation in GRASS manual for QGIS users.
- more intuitive distribution of parameters in GRASS - e.g. 'r.slope.aspect' could be divided into tabs slope and aspect, after this is not necessary to divide module in Processing Plugin to two modules - one for slope and second for aspect. Because it is intuitive to use just one tab if I want to compute slope.
