GSoC 2017 SOS tools in GRASS GIS

Title:	SOS tools in GRASS GIS
Student Name:	Ondrej Pesek, ​Czech Technical University in Prague
Organization:	​OSGeo - Open Source Geospatial Foundation
Mentor Name:	​Luca Delucchi, Matteo De Stefano
GSoC proposal:	​view proposal
Repositories:	Github for development:	​https://github.com/pesekon2/GRASS-GIS-SOS-tools
	GRASS SVN AddOns for final modules:	​https://svn.osgeo.org/grass/grass-addons/grass7/
	OWSlib fork with SOS patches:	​https://github.com/pesekon2/OWSLib
	istSOS fork with some patches:	​https://github.com/pesekon2/istsos2

Abstract

GRASS GIS doesn’t have any module to work with Sensor Observation Service (SOS). SOS is an Open Geospatial Consortium (OGC) standard and it is useful to implement modules to work with this standard into GRASS.

Goal

Intended modules would enable the user to create a vector with or without the observation values, create a raster for each queried day and create a space time vector or raster dataset. One module would also allow the user to convert a space time vector dataset into a raster dataset. The user should be also allowed to get the capabilities to get info about sensors from these modules and filter the results.

Intended modules would also use OWSLib to work with the server and data. There are a few things to improve in OWSLib, so this should be the first part of my GSoC project. I would also like to use pyGrass (a python library allowing users to access the low-level GRASS API) to create new modules.

Timeline

		status
Community bounding period	Studying OWSLib and SOS	*
Let the summer begin
MAY 30 - JUNE 2	Testing OWSLib functionalities on different SOS servers	*
JUNE 5 - JUNE 9	Improving OWSLib functionalities	*
JUNE 12 - 16	Improving OWSLib, developing background for forthcoming modules	*
JUNE 19 - 23	v.in.sos (creating a vector with or without the observation values)	*
JUNE 26 - 30	v.in.sos upgrades, t.vect.in.sos shuck	*
First evaluations
JULY 3 - 14	I am sorry but I will have no access to internet and neither to electricity due to my work at summer camp	*
JULY 17 - 21	t.vect.in.sos (creating a space time vector dataset)	*
JULY 24 - 28	r.in.sos (creating a raster for each queried day)	*
Second evaluations
JULY 31 - AUGUST 4	t.rast.in.sos (creating a space time raster dataset)	*
AUGUST 7 - 11	t.vect.to.rast (convert space time vector dataset into space time raster dataset)	*
AUGUST 14 - 18	Writing documentation, buffer in case of delay
AUGUST 21 - 29	Submitting the final work product
Final evaluations

Requirements

​GRASS 7.3

Development

Weekly reports

May 30 - June 2

Studied OWSLib and its work with SOS (especially with istSOS) and in which formats are available in response.

Solved ​this issue in OWSLib.

Created converter between normal JSON output from OWSLib and geoJSON for points

June 5 - June 9

I had a problem with xml parser in OWSLib for SOS observations, because it works completely different way than we expected. After few conversations I have decided to work with raw output and write parser by myself.

Created pull request solving ​encoding issue for OWSLib.

Created converter between text/xml;subtype="om/1.0.0" output from OWSLib and geoJSON for points

Created a shuck of v.in.sos module

June 12 - June 16

Created v.in.sos

• Makefile and first html
• UI
• Prints informations about sensor offerings
• Imports the observation into GRASS GIS
• This version is returning just the last observation

June 19 - June 23

v.in.sos

• flags for printing timestamps of begin and end of observation
• working with event_time (interval of observation is used in query)
• v.in.sos returns one map with many layers (layers count = observed properties count)

xml2geojson

• working with separated blocks of values, not just with single values

June 26 - June 30

v.in.sos

• using vectors in pyGRASS instead of v.in.sos
• support multiple offerings
• layers are for offerings and observed properties, rows are for one feature
• handle data input with no observations for
• handle too big input with warnings and errors

t.vect.in.sos

• first version (just draft with some bugs, they are saved as issues and will be solved after my return)

July 3 - July 7

I was out of internet and electricity during the last week. So I wasn't able to do anything connected with GSoC.

July 10 - July 14

I was out of internet and electricity during the last week. So I wasn't able to do anything connected with GSoC.

July 17 - July 21

Visited FOSS4G-E

Implemented shell script style output into modules

t.vect.in.sos

• new vector for each offering and observed property

r.in.sos

• first version

July 24 - July 28

All modules

• Shell script style update
• Handle import errors
• Flags working with multiple offerings
• Not given options are generated just temporarily

xml2geojson

• Observed properties can be given as their ids

t.vect.in.sos

• Creating stvds

r.in.sos

• Creating raster maps

July 31 - August 4

r.in.sos

• flag for deleting vector maps created as intermediates
• importing vectors and values in one step

v.in.sos

• importing vectors and values in one step

All modules

• not stop when using just -g flag
• code cleaning
• manuals updates

A little pull request for istSOS

• allow user to define offering in csv2istsos

August 7 - August 11

v.in.sos

• finally working the desired way even with more layers
  • uses yet existing columns for timestamps instead of creating new ones

t.vect.in.sos

• finally working with layers-based vectors

t.rast.in.sos

August 14 - August 18

all modules

• made quiet, e.g. when creating temporal dataset with hundreds of maps, it doesn't show hundreds of builds
• worked on documentation

t.vect.in.sos

• uses yet existing timestamped layers instead of creating new ones

v.to.rast

• created

r.in.sos

• updating yet existing maps instead of creating new ones
• option for granularity and aggregations