wiki:SampleDataset

Table of Contents

  1. Proposal
    1. Related other locations
    2. Basic NC location with PERMANENT mapset
      1. Common rules
      2. Rasters
      3. Vectors
    3. Specialized Mapsets
      1. Elevation data
      2. Remote sensing
      3. Vector network
      4. Point data
      5. Time series
  2. Notes after HVA discussion with Hamish in 2014
  3. Comments
  4. Notes about data sources
  5. TODO
  6. Integration with GRASS GIS unit test suite
  7. Download of draft location package

GRASS GIS Sample Datasets

Description and discussion of (future) standardized GRASS GIS 7 dataset - an updated, simplified and extended nc_spm_08, to be used in manual pages, tutorials, courses, code testing, development and OSGeo Live.

Note: The user-oriented page with examples and list of existing datasets is at ​GRASS Wiki: GRASS GIS Standardized Sample Datasets.

Contributors: Helena Mitasova, Markus Neteler, Vaclav Petras, Anna Petrasova, Hamish Bowman, [contribute and add yourself]

Proposal

The complete sample data set will include:

  • ncarolina_spm_complete location with PERMANENT mapset with baseline data and several additional mapsets with specialized data in NC State plane [m] (see ​North Carolina - EPSG codes)
  • world_ll location in geographic coordinate system [deg]
  • external data in various coordinate systems

Related other locations

Basic NC location with PERMANENT mapset

This is based on the original nc_spm_08 location with simplified, standardized names for map layers, some updates. To keep the data set simple, more specialized files were moved into separate mapsets:

  • ncarolina_spm_base
    • PERMANENT
      • raster
      • vector
    • practice1 (empty mapset)

Common rules

  • Names of maps/layers must be the same for all standardized datasets. No additions to names such as _10m or _wake_county are allowed. This also implies that names must be in English, national language is not allowed for national standardized datasets (however, if desired, we can can work on a script which would automatically ​rename multiple maps in dataset and would also find and replace names in documentation and tutorials).
  • Description in tables here should be usable as title of the map. Separate details in description, which should not be part of the title, using commas or parentheses. Titles can differ between standardized datasets and can use national language (unlike names).

Rasters

  • resolution and extent (in the sense of number of rows and columns) should be the same of very similar for all standardized datasets (obviously, actual geographical extent can be different)
  • standard resolution: 10m (marked as std)
  • standard rows x columns (cells): xxx (x) (marked as std)
name area for NC description for NC resolution rows x columns (cells) note
basins South-West Wake county Watersheds derived from NED 30m std std
elevation South-West Wake county Elevation NED std std
elevation_shade South-West Wake county Shaded relief std std
geology South-West Wake county Geology derived from a vector map std std
lakes South-West Wake county Wake county lakes std std
landuse South-West Wake county Landuse in 1996 std std
orthophoto for CC or rural area orthophoto (R, G, B, NIR) 1m res or better
soils South-West Wake county soil type should be vector data?

Vectors

name area for NC description for NC feature type number of features note
boundary_region South-West Wake region boundary polygon map
boundary_state State of NC NC State map polygon map
census Wake County Wake County census blocks with attributes, clipped polygon must have population data
firestations Wake County fire stations points map
geology Wake County North Carolina geology map polygon map
geonames Wake County geonames points map
hospitals Wake County North Carolina hospitals point map
history_markers Wake County point map move to archeology mapset?
parcels for CC or rural area point map
points_bare_surface CC or rural bare ground lidar points for interpolation point map
points_of_interest Wake County points of interest (examples?) point map
railroads North Carolina railroads lines map
roadsmajor Wake County major highways and roads lines map
schools Wake County schools points map
streams South-West Wake streams lines map
streets Wake County roads and streets lines map
zipcodes Wake County zip codes polygon map

Specialized Mapsets

To be distributed with the ncarolina_spm_base location so that they include PROJ information and are readable by GRASS. Note: Mapsets cannot be distributed without a location because they lack PROJ_INFO (in our experience providing solely mapsets did not work, it was very confusing for users. But perhaps the only problem was that that "GRASS GIS 7.0 welcome screen" cannot unpack (unzip, untar) a mapset and copy it into an existing mapset. (MN: we need a button "Download sample data" in the welcome screen!)

Elevation data

  • elevation: several elevation models at different scales, lidar

Remote sensing

Vector network

  • networks: vector networking data + LRS
  • orthoimg: set of aerial image scenes for image classification, including images from UAVplain

Point data

  • archeology: sites? historical maps/topography
  • meteo stations

Time series

Notes after HVA discussion with Hamish in 2014

Helena, Vaclav, Anna and Hamish at GSoC 10 Year Reunion / Mentor Summit in November 2014

  • barebone dataset for OSGeo Live which will include a script to generate the derived data needed for tutorials (?)
  • metadata
  • maps (layers)
    • exclude layers which can be generated
    • secref elevation? + orthophoto
    • add NC WMS service to GUI
    • soils, geology, lakes just vector
    • elevation, landuse, orthophoto
    • fields - parcel plots - with anonymized names - for secref - planimetry
    • SPOT image
    • elevation and precipitation points into baseline
    • add zipcodes
    • powerlines (extended)
    • separate layers for state and counties boundaries
    • natural earth for latlon dataset (data are on ​github)

Comments

The data sets can be distributed separately or we can have packages with several mapsets or all mapsets, depending on the size. I found that packaging and distributing mapsets without location is not practical, so I ended up distributing the specialized mapsets with nc_spm_baseline or world_baseline - is this OK?. We also need to figure out how to include the original metadata that come with the original data - link to the source in history file may be enough.

The baseline location+mapset should be simple with easy to understand names of map layers, my only issue is the loc_ncspm_baseline name as I am not able to come up with a simple name that would say that this is a location with North Carolina data in state plane meters coordinate system - maybe loc_ncarolina would be better, assuming that state plane in meters is the official coordinate system for NC? But I also keep ncspf for feet and ncutm.

Notes about data sources

There is a lot of data, main challenge now is to select a consistent, meaningful, but not too large data set. Many data sets are regularly updated and new ones are posted but tutorials and man pages require stable data to work the history file should include link to the original data source with a note that an updated version of the data map be available from there

Significant natural heritage areas and natural heritage element occurrences posted on NC one map.

TODO

List of actionable items.

  • replace historical markers with historical places (done?)
  • Keep in mind what tasks we want to do with the data when selecting them (e.g., table join, selections, buffering etc). This influences the choice.

Integration with GRASS GIS unit test suite

All tests in the "gunittest" environment (see also ​overview) need to be written in a way that the map names correspond. See also

​http://grass.osgeo.org/grass71/manuals/libpython/gunittest_testing.html#data

Download of draft location package

Last modified 9 years ago Last modified on 03/04/16 03:20:55
Note: See TracWiki for help on using the wiki.