RPC interface for persistent GUI

Proposal of a RPC interface for persistent GUI applications.

Introduction

Like any other GIS needs GRASS a reliable graphical user interface (GUI) to manage, display and interact with maps. The GRASS GIS core libraries should be used to implement a GUI to achieve good performance, especially in case of vector editing. The design of the GRASS core libraries targets the implementation of GRASS modules and is not designed for persistent applications like a GUI. The main problem for persistent application is, that several library functions will terminate the application in case of a fatal error. This approach makes the implementation of GRASS modules more easy, but is not applicable in persistent applications, since the handling of fatal errors is mostly done by library functions.

One reasonable approach to use the GRASS GIS core libraries in persistent applications is the implementation of a Remote Procedure Call wrapper, that provides an interface to GRASS library functions that exists in a different process space than the persistent application. This will avoid the termination of the persistent application in case of a fatal error. The drawback is the complex implementation of a RPC interface, or in case a third party solution is used to implement the RPC interface additional dependencies. The RPC interface will also add more overhead to each function call, since data that is passed between persistent application and the GRASS library interface process must be serialized and transferred via Interprocess Communication (IPC). It will also add more code and complexity to GRASS that must be maintained.

The benefits of a RPC interface are:

• The ability to implement fast and well function persistent client applications like GUI's
• Support for different programming languages on client side (Python, C++, Java, JavaScript) that will have access to GRASS library functions
• Implementation of GUI specific functionality, hence bundling of GRASS library functions on the server side
• Support for parallel reading of raster and vector maps using several server processes in parallel
• Providing a well designed object oriented interface to GRASS that will not change in case the underlying GRASS core libraries change
• The possibility that none-GPL applications can directly access GRASS library functions without suffering from the viral Gnu Public License that is used in GRASS (well that depends on the licensing of the RPC interface as well)

Implementation

I would strongly suggest to use apache thrift[1] to implement the RPC interface for GRASS core libraries. This framework support plenty of programming languages, adds minimal dependencies and provides platform independent RPC functionality (data serialization, IPC).

The RPC interface will not only do simple function wrapping, it will also implement a higher level interface:

• Map metadata read access as single client function will do on the server side: open map; read metadata; close map; send data

• Fast raster and vector map read access as single client function will do on server side: open map; read requested map into memory based on bounding box and render resolution; close map; send data

• Vector editing: keep a single vector map in update state open in a dedicated grass vector editing process. In case of a fatal error, this map may be lost or corrupted. It is important that this process will not be used for other purposes than single vector map editing.

• Vector analysis functionality that need to keep vector maps open in read only state for fast topological access. Information's about the open vector maps (i. e. the position of the next line to be read) may be lost in case the server process terminates.

• Same for raster map analysis

The client will detect if the server was terminated and will raise an exception.

The RPC interface should only support meaningful functions to be used in a GUI, that's its only purpose. It should not be used to implement processing algorithms or GRASS modules. But it will in addition provide capabilities to use several processes to read different chunks of the same map in the GUI, hence parallel read only map access. This can be nicely done using a pool of GRASS server processes.

[1] ​https://thrift.apache.org/

Data Exchange

First we must determinate what data structures should be exchanged between server and client applications (this proposal is thrift specific):

• Strings, integer, double and boolean types as argument for functions and return values
• Map metadata (raster, vector, voxel), region information, projection information as specific thrift structures using strings internally
• Time stamps as specific thrift structure using integers, strings and double types
• Raster data (single row and whole map) as byte array (RGB map), integer array (CELL type) and double array (FCELL and DCELL)
• Vector data (single line/feature) as specific thrift structure with metadata (line type, ...), integer array for line_cats and double array for line_points
• Several vector lines as ordered list using the specific thrift structure
• Vector data (single line/feature) as well known binary format stored in a binary array
• Vector data (whole map) as well known binary format (wkb) stored in a binary array
• Voxel data (single value, tile and whole map) as byte array (RGB map) and double array (FCELL and DCELL)
• A hash table (map) structure with internal string, integer and double arrays that represent single columns of a database table

Higher level API

The server process should be able to keep maps open in read only mode, to allow fast access to raster and vector data for analysis. Maps are identified by their name@mapset id, hence a hash table will be used to store the file descriptors and map structures in the server process. Hence open and close map functions must be provided by the RPC interface.

A single vector map should be kept open in editing mode for vector editing purposes.

Proposed higher level functions:

• Read a whole raster map as RGB array, integer array or double array using a specific extent and resolution informations
• Read a whole vector map as wkb
• Read a portion or a whole vector map attribute table
• Get all metadata for vector, raster and voxel maps (spatio-temporal extent, resolution, ...) using a single function call
• Call a GRASS module, using the server process specific location/mapset and region settings
• List all available maps (raster, vector and voxel)