opendatacube jupyter notebooks

[darkblueb]

ol14 try these:

​https://github.com/ceos-seo/data_cube_notebooks/blob/master/docs/notebooks_overview.md

[kalxas]

Ticket retargeted after milestone closed

[darkblueb]

review for ol15

​https://wiki.osgeo.org/wiki/Osgeolive-jupyter

[darkblueb]

Is it possible to make a dot-deb package of py3 module ​-LINK-

lark_parser-0.12.0-py2.py3-none-any.whl

ref

​https://wiki.osgeo.org/wiki/Osgeolive-jupyter#opendatacube

[darkblueb]

attempting a setup with datacube installed

​http://ppa.launchpad.net/osgeolive/nightly/ubuntu/pool/main/d/datacube/

when I make a call on the command line to datacube tool, I get rasterio.errors.CRSError: The EPSG code is unknown. PROJ: proj_create_from_database: Cannot find proj.db

inline in ipython3, I can import individually rasterio, datacube, pyproj with no errors

I manually set the bash shell environment variable $ DATACUBE_CONFIG_PATH=~/.config/datacube/datacube.conf datacube product add landsat-clip.yaml

​https://proj.org/usage/environmentvars.html#envvar-PROJ_LIB

those files are in /usr/share/proj/

the error looks a lot like this ​https://rasterio.readthedocs.io/en/latest/faq.html#why-can-t-rasterio-find-proj-db-rasterio-versions-1-2-0

what is odd is that rio and inline rasterio work fine

more details here:

​https://wiki.osgeo.org/wiki/Osgeolive-jupyter#opendatacube

[kalxas]

lark-parser 0.12.0 is an older version of lark 1.1.1 that is available upstream on Ubuntu.

OpenDataCube needs to update to the latest version at some point, until then I have prepared a deb package of the old version.

[darkblueb]

ol15 build 167s

user@osgeolive:~$ ./do_setup_datacube.sh 
CREATE EXTENSION
CREATE EXTENSION
2022-08-04 04:52:11,495 8363 datacube INFO Running datacube command: /usr/bin/datacube -v system init
Initialising database...
2022-08-04 04:52:11,991 8363 datacube.drivers.postgres._core INFO Ensuring user roles.
2022-08-04 04:52:11,997 8363 datacube.drivers.postgres._core INFO Creating schema.
2022-08-04 04:52:11,998 8363 datacube.drivers.postgres._core INFO Creating tables.
2022-08-04 04:52:12,028 8363 datacube.drivers.postgres._core INFO Creating triggers.
2022-08-04 04:52:12,032 8363 datacube.drivers.postgres._core INFO Creating added column.
2022-08-04 04:52:12,033 8363 datacube.drivers.postgres._core INFO Adding role grants.
2022-08-04 04:52:12,035 8363 datacube.index.postgres.index INFO Adding default metadata types.
Created.
Checking indexes/views.
2022-08-04 04:52:12,217 8363 datacube.drivers.postgres._api INFO Checking dynamic views/indexes. (rebuild views=True, indexes=False)
Done.
Adding "clip_landsat" (this might take a while) DONE

user@osgeolive:~$ cat do_setup_datacube.sh 
#!/bin/sh

set -e

createdb datacube
psql datacube -c 'create extension postgis'
psql datacube -c 'create extension hstore'

# reset
# rm -rf .config/datacube
# psql -c 'drop database datacube'

##-------------------------------------------------
USER=user
DCONF=/home/${USER}/.config/datacube

mkdir -p ${DCONF}

export DATACUBE_CONFIG_PATH=${DCONF}/datacube.conf

#----------------------------------------------
cat << EOF > ${DCONF}/datacube.conf
[datacube]
db_database: datacube

# A blank host will use a local socket. Specify a hostname (such as localhost) to use TCP.
db_hostname:

# Credentials are optional: you might have other Postgres authentication configured.
# The default username otherwise is the current user id.
db_username: user
db_password: user
EOF

##---------------------------------------------

cat << EOF > ${DCONF}/landsat-clip.yml
name: clip_landsat
description: example ortho imagery
metadata_type: eo3

license: CC-BY-4.0

metadata:
  product:
    name: clip_landsat

storage:
  crs: EPSG:32619
  resolution:
    longitude: 28.497418829384827
    latitude: -28.503958771796388

measurements:
  - name: imagery
    dtype: uint8
    nodata: -32768.0
    units: "metre"

EOF

##-------------------------------------------------

datacube -v system init

datacube product add ${DCONF}/landsat-clip.yml

[kalxas]

Excellent progress Brian!

Can you please draft an install_opendatacube.sh with the above and make a pull request in the labs folder?

[darkblueb]

ol15 build 179

user@osgeolive:~$ pip3 install --upgrade-strategy=only-if-needed --user odc-stac
Collecting odc-stac
  Downloading odc_stac-0.3.1-py3-none-any.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.1/53.1 KB 400.8 kB/s eta 0:00:00
Requirement already satisfied: rasterio in /usr/lib/python3/dist-packages (from odc-stac) (1.3.0)
Requirement already satisfied: pandas in /usr/lib/python3/dist-packages (from odc-stac) (1.3.5)
Requirement already satisfied: numpy in /usr/lib/python3/dist-packages (from odc-stac) (1.21.5)
Requirement already satisfied: affine in /usr/lib/python3/dist-packages (from odc-stac) (2.3.0)
Collecting xarray>=0.19
  Downloading xarray-2022.6.0-py3-none-any.whl (927 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 927.2/927.2 KB 393.8 kB/s eta 0:00:00
Collecting odc-geo>=0.1.3
  Downloading odc_geo-0.2.1-py3-none-any.whl (102 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.8/102.8 KB 398.9 kB/s eta 0:00:00
Collecting pystac<2,>=1.0.0
  Downloading pystac-1.6.1-py3-none-any.whl (146 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 146.0/146.0 KB 408.3 kB/s eta 0:00:00
Requirement already satisfied: toolz in /usr/lib/python3/dist-packages (from odc-stac) (0.11.2)
Requirement already satisfied: dask[array] in /usr/lib/python3/dist-packages (from odc-stac) (2022.1.0+dfsg)
Requirement already satisfied: shapely in /usr/lib/python3/dist-packages (from odc-geo>=0.1.3->odc-stac) (1.8.0)
Requirement already satisfied: pyproj in /usr/lib/python3/dist-packages (from odc-geo>=0.1.3->odc-stac) (3.3.0)
Requirement already satisfied: cachetools in /usr/lib/python3/dist-packages (from odc-geo>=0.1.3->odc-stac) (5.0.0)
Requirement already satisfied: python-dateutil>=2.7.0 in /usr/lib/python3/dist-packages (from pystac<2,>=1.0.0->odc-stac) (2.8.1)
Requirement already satisfied: packaging>=20.0 in /usr/lib/python3/dist-packages (from xarray>=0.19->odc-stac) (21.3)
Installing collected packages: xarray, pystac, odc-geo, odc-stac
Successfully installed odc-geo-0.2.1 odc-stac-0.3.1 pystac-1.6.1 xarray-2022.6.0

[kalxas]

Thanks for the feedback, it won't be very hard to create packages for odc_stac and odc_geo. On my todo.

[kalxas]

Done

​https://launchpad.net/%7Eosgeolive/+archive/ubuntu/nightly/+packages?field.name_filter=&field.status_filter=published&field.series_filter=jammy

[kalxas]

​https://github.com/OSGeo/OSGeoLive/commit/5c89ae3167ed87a799f831f416f4dcfa5847ba3e

[darkblueb]

repos

​https://github.com/digitalearthafrica/deafrica-sandbox-notebooks.git

docs

​https://docs.digitalearthafrica.org/en/latest/sandbox/notebooks/Frequently_used_code/Analyse_multiple_polygons.html

[kalxas]

Ticket retargeted after milestone closed

[darkblueb]

fast-forward some weeks -- the sandbox noted above is not maintained by anyone at this time. PR for a clean datacube init on github now.

work-in-progress

​https://github.com/OSGeo/OSGeoLive/pull/367

[darkblueb]

base install, Overview, Quickstart, first Notebook.. checked in to git

[darkblueb]

included in ol16 beta1