[[TOC]] = GSoC 2014 Metadata for GRASS GIS maps and datasets = || Title: || '''Metadata for GRASS GIS maps and datasets'''|| ||Student Name: || Matej Krejci, [http://www.cvut.cz Czech Technical University in Prague]|| ||Organization: || [http://www.osgeo.org OSGeo - Open Source Geospatial Foundation]|| || Mentor Name: || [http://grass.osgeo.org/wiki/User:Landa Martin Landa], [http://grass.osgeo.org/wiki/User:Madi Margherita Di Leo] || || GSoC proposal: || [http://www.google-melange.com/gsoc/proposal/public/google/gsoc2014/krejcmat/5629499534213120 view proposal] || || Link to code: || [http://trac.osgeo.org/grass/browser/grass-addons/grass7/gui/wxpython/wx.metadata] || Documentation || [http://grasswiki.osgeo.org/wiki/ISO/INSPIRE_Metadata_Support] = Motivation = Current GRASS GIS offers just a poor and out-of-date administration of metadata. The goal of this project is to implement administration of metadata according to the standards of ISO 19115, ISO 19139 and to create a graphical user interface which would allow the user to operate with metadata, such as creating, updating, browsing, searching etc. = Timeline = |||| timeline from proposal ||newly proposed timeline (1)||status (1)|| after mid-term proposed timeline (2)||status (2)|| ||MAY 1- 7 || "Early starts" Study ISO, INSPIRE and FGDC standards ||||+|| |||| ||MAY 7 - 14 || Getting familiar with GRASS GIS modules code. ||||+|||||| ||MAY 14 - 19 || Design of metadata GRASS library|| design GRASS library (in Python) to read / write metadata based on ISO as flat XML files||+|||||| ||MAY 19 || Starting GoSC || || || || || ||MAY 19- 25 || Support of different metadata profiles ||design GRASS library (in Python) to read / write metadata based on ISO as flat XML files including profies (XSD validation, Inspire as the show-case)||+|||||| ||MAY 26 - JUN 16 || "Break"(Prepare for bachelor final exam) ||add support for conversion GRASS (current) metadata files <-> XML ISO metadata ||+|||||| ||JUN 16 - 24 || Implement basic library model for command line ||design cmd modules for reading and writing ISO-based metadata -> r.info.iso + v.info.iso (including support for conversion GRASS (current) metadata files <-> XML ISO metadata)||+|||||| ||JUN 24 - 30 || Modules functionality (searching, publishing metadata)|| design GUI for reading and writing ISO-based metadata|| ||design source of information from jinja||+|| ||JULY 1 - 7 || Design GUI|| design GUI for reading and writing ISO-based metadata -> g.gui.metadata|||| g.gui.metadata version 1.0||+|| ||JULY 7 - 14 || Connect GUI library, basic functions|| advanced metadata publishing/searching based on owslib + pycsw|||| g.gui.metadata version 1.1||+|| ||JULY 14 - 21 || Implementation of search function to GUI ||advanced metadata publishing/searching based on owslib + pycsw|||| g.gui.metadata version 1.2 ||+|| ||JULY 21 - 30 || Improve metadata validation using designed GUI || advanced metadata publishing/searching based on owslib + pycsw -> testing prototype|||| g.gui.metadata 1.3&1.4., polishing and testing g.gui.metadata ||+|| ||AUGUST 1 - 9 || Polishing code|| polish code and advanced metadata publishing/searching based on owslib + pycsw, g.gui.metadata 1.5|||| designing of metadata catalog browser||+-|| ||AUGUST 9 - 18 || Testing and documentation |||| ||||+|| ||AUGUST 18 || The end of GSoC |||| |||||| = Development = == Requirements == [http://grass.osgeo.org/download/software/#g71x GRASS 7.1] == Dependencies == Current development of metadata management has dependence on external libraries: * OWSLib (no additional dependence) development version () {{{ git clone git://github.com/geopython/OWSLib.git cd OWSLib && sudo python setup.py install }}} * Jinja templates {{{ git clone git://github.com/mitsuhiko/jinja2.git cd jinja2 && sudo python setup.py install }}} == Architecture == The new designed metadata management in GRASS is based on external python modules without processing of the GRASS core. * First step which is partly done is to create basic modules for exporting ISO based metadata for raster and vector maps. The modules are intended to implement class that is providing parsing metadata from r.info and v.info. These metadata are assigned to ISO attributes with using OWSLib. The OWSLib is offering creation instances of ISO elements. The instances are suitable to filled predefined of Jinja xml ISO profiles. From this part of project r.info.iso and v.info.iso are created. === r.info.iso and v.info.iso === * These modules are based on the mentioned implementation. * Offers converting metadata from *.info to xml file based on ISO standard. * Currently available metadata profile for GRASS: * INSPIRE profile, * GRASS BASIC profile - this profile is mainly including available metadata from GRASS. === g.gui.metadata === Current task is to design and the implementation of the GUI interface. Below is the first design provided. Main aggravating factor is, that some attributes in ISO can be defined multiple times. For the solution, there is necessary to create interface for getting information from some source. Currently the easiest way is to get information forom predefined XML Jinja template. Interface ensures independence between editor and source of information. This is useful for future development steps. The editor will support: || version || function || || 1.0 || to implement main window and layout || || || function for browsing maps with using treeCtrl || || || to create xml ISO file (with using v/r.info.iso) || || || interface to get information about ISO attributes || || 1.1 || editing of created xml files with using Jinja templates || || 1.2 || to add multiple values for specific attributes || || 1.3 || validator of inspire xml || || 1.4 || multiple metadata editing ( choosing maps invokes process that changes selected attributes of all chosen maps. || || 1.5 || connection with r.support and v.support. grass ISO profile validator || scheme of designed implementation [[Image(diagram.png)]] draft of GUI for choosing ISO profile or options edit xml only [[Image(main.png)]] draft of GUI editor [[Image(prototyp1.png)]] == How to test == {{{ GRASS> g.extension wx.metadata GRASS> r.info.iso elevation GRASS> g.gui.metadata }}} == Weekly reports == === Week 1 (May 23) === studied metadata and tried to find suitable python lib during last three weeks: * studied ISO 19115 and EN ISO 19119 * INSPIRE * be advised to leave out implementation of FGDC - just ISO, INSPIRE * get familiar with OWSLib and pycsw * set pycsw metadata server catalog * handled metadata by OWSLib * spent time with design lib based on automated generation of XML from XSD - failed * tested suitability of generateDS to generate python object from XML scheme (XSD) * designed library for parse GRASS GIS metadata to XML based on md. ISO * parsed metadata from: * r.info * v.info * designed essential metadata profile for raster and vector maps (based on ISO standard) * prepared lib for new update of OWSLib that will be ready during next week. (in contact with author) * created new timeline - main changes: * will create basic metadata management based on creating, editing, validating ISO. (obligatory) * will create advanced management based on client-server with using pycsw. (optional) [http://lists.osgeo.org/pipermail/grass-dev/2014-May/069086.html report email] === Week 2 (May 30) === created cmd modules for writing ISO-based metadata -> r.info.iso + v.info.iso support: * basic grass ISO profile * inspire profile (for subsequent editing in future wxGUI editor) [http://lists.osgeo.org/pipermail/grass-dev/2014-May/069220.html report email] === Week 3 (June 06) === * Consulted design of metadata editor with mentors. * Consulted current implementation of v.info.iso and r.info.iso * Search for alternatives to Jinja templates (current implementation) * Learned basics of wxPython [http://lists.osgeo.org/pipermail/grass-dev/2014-June/069374.html report email] === Week 4 (June 13) === [http://lists.osgeo.org/pipermail/grass-dev/2014-June/069502.html report email] === Week 5 (June 22) === Redesigned GUI of editor with inspiration from [http://inspire-geoportal.ec.europa.eu/editor/ INSPIRE Editor] - The editor will include: * on the top (fixedly) = toolbar: create new, save, validate, etc (new) * panels based on MultiSplitterWindow * on the left side = panel with Grass map browser (proposed) * on the right side ='notebook' with two items: advanced browser/editor and html help (new) * in the middle = Main editor of metadata Implemented part of GUI editor: * part of the GUI (main frame and layout) based on MultiSplitterWindow * GRASS maps browser in the left panel * in right panel implemented the "notebook" with metadata browser and editor (necessary to tune up the view of metadata). * class with icon based toolbox [[Image(md_editor1.png)]] [http://lists.osgeo.org/pipermail/grass-dev/2014-June/069619.html Report email] === Week 6 (June 29) === Worked on GUI editor * implemented generator of metadata items (control text, label, button) in main editor * fixed some bugs in g.gui.metadata * implemented interface for parse information from jinja template * added additional information about metadata to jinja template. [[Image(gui.png)]] [http://lists.osgeo.org/pipermail/grass-dev/2014-June/069826.html Report email] === Week 7 (July 4) === * designed and implemented function that allows generating GUI editor from xml loaded by OWSLib with using objects and loops from Jinja template. [[Image(gui1.png)]] [http://lists.osgeo.org/pipermail/grass-dev/2014-July/069946.html Report email] === Week 8 (July 11) === * re-implemented generator of GUI to more object oriented design * implemented main part of exporting edited metadata to XML * started with dynamically adding wx widgets to frame (multiplicity of metadata items) * changed timeline [http://lists.osgeo.org/pipermail/grass-dev/2014-July/070005.html Report email] === Week 9 (July 18) === * implemented 'multiplicity button' for dynamically adding widgets and also removing them * support of exporting metadata from dynamically generated GUI(multiplicity button) (need more working on allow exporting keywords to xml(multiple keywords)) * connected finished modules together(g.gui.metadata) * validator * tree editor * main editor * toolbox [[Image(g.gui.metadata1.jpg)]] === Week 10 (July 25) === [http://lists.osgeo.org/pipermail/grass-dev/2014-July/070214.html Report email] === Week 11 (August 1) === The yellow text inputs marking the values which are not possible to fill by information from r.info and v.info. On the picture below is the session of editor which allows to mark metadata items for predefining template and save it for next times. This function is well fit to editing multiple maps with batch exporting. Red colors marking the metadata which are obligatory for chosen jinja template but the loaded xml(ISO metadata) doesn't include them. [http://lists.osgeo.org/pipermail/grass-dev/2014-August/070310.html Report email] [[Image(templateCreator.png)]] The editor allows editing maps from GRASS which GRASS BASIC profile and INSPIRE profile. Also allows to initialize editor with custom templates which will be filled as much as possible by intersection with metadata in selected maps (intersection of inspire template and own template). Of course editor can be initialize just with template and xml without any connection to GRASS mapset. [[Image(multiSel.png)]] === Week 12 (August 8) === [http://lists.osgeo.org/pipermail/grass-dev/2014-August/070348.html Report email] === Week 13 (August 15) === [http://grasswiki.osgeo.org/wiki/ISO/INSPIRE_Metadata_Support see documentation on wiki page] [http://lists.osgeo.org/pipermail/grass-dev/2014-August/070427.html Report email] == TODO&IDEAS == * Stored predefined metadata values in SQLite i.g.: keyword lists, points of contacts, and so on.[http://lists.osgeo.org/pipermail/grass-dev/2014-July/069872.html mailing list] * to allow choosing topics from list(combobox) * export metadata to html->pdf * tooltips for toolbar * renaming metadata according to renaming maps * bug? http://thehacklist.blogspot.cz/2010/06/resolving-murrinestyledrawbox-assertion.html * bug - gives empty space in the field of abstract * renaming metadata according to renaming maps in GRASS * change BB to WGS84 by default * The bounding box shall be expressed in decimal degree with a precision of at least 2 decimals * The coordinates of the bounding box are expressed in any geodetic coordinate reference system with a Greenwich Prime Meridian * to connect main validator with text inputs validation(email,number etc.)