wiki:GSoC/2021/JupyterAndGRASS

Version 15 (modified by chaedri, 3 years ago) ( diff )

--

GSoC 2021 Improved Integration of GRASS GIS and Jupyter Notebooks

Title: Improved Integration of GRASS GIS and Jupyter Notebooks
Student Name: Caitlin Haedrich, North Carolina State University
Organization: OSGeo - Open Source Geospatial Foundation
Mentor Name: Vaclav Petras, Helena Mitasova, Stephan Blumentrath
GitHub Repository: View Repo
GSoC proposal: View Proposal

Abstract

The current integration of GRASS GIS and Jupyter Notebooks requires a cumbersome environment variable setup after launching GRASS from within the notebook. It also only allows for simple, non-interactive map displays. My project will address both these issues and improve the GRASS-Jupyter integration with a new package called grass.jupyter.

Goal

My goal is to design a Python/PyGRASS library which will have 3 main components for improving the GRASS-Jupyter integration: (1) creating new initiation functions for the launch of GRASS GIS in Jupyter Notebooks, (2) creating functions for more intuitive map display and, (3) introducing an interactive map display function.

Timeline

Time Period

Milestones

Tasks

Status

May 17th - June 7th
Community Bonding

  • Introduce myself in dev and SOC mailing lists
  • Get in contact with mentors and discuss project
  • Prepare the wiki page
  • Set up the GitHub repository for project
  • Set up developer environment

Ok
Ok
Ok
Ok
Ok

June 7th - June 11th
Week 1

  • Finish Binder Set Up
  • Write grass.jupyter initiation functions

Ok
Ok

June 14th - June 18th
Week 2

  • Write non-interactive display functions

Ok

June 21st - June 25
Week 3

  • Begin writing interactive/folium map functions
  • Edit non-interactive display functions

Ok
Ok

June 28 - July 2
Week 4

  • Continue writing interactive/folium map functions for vectors

Ok

July 5th - July 9th
Week 5

  • Continue writing interactive vector functions
  • Begin writing interactive raster functions

Ok
Ok

July 12th - July 16th
Week 6: Evaluations

  • Receive feedback and write evaluation
  • Switch to temporary files and directories
  • Continue working on folium functionality

Ok
Ok
Ok

July 19th - July 23rd
Week 7

  • Improve documentation and error handling
  • Draft test module for non-interactive rendering
  • Continue raster functions for folium

Ok
Ok
Ok

July 26th - July 30th
Week 8

  • Finish draft of raster methods with folium
  • Improve test module

Ok
Ok

August 2nd - August 6th
Week 9

  • Finish Documentation and tutorials

August 9th - August 13th
'Week 10'

  • Write project summary

August 16th- August 23rd
Week 11: Final Evaluation and Code Submission

  • Submit code and final evaluation.

Bonding period report

With the bonding period wrapped up, here is a brief update on what I've been working on so far.

  1. What did I accomplish during the bonding period?
  • I introduced myself in dev and SOC mailing lists (1).
  • I had a productive meeting with my mentors, Vashek Petras, Helena Mitasova and Stefan Blumentrath last Friday. We discussed the best development environment (see second to last bullet point) and setting up the main GRASS repository to run in Binder (see last bullet point).
  • Prepared the wiki page.
  • Set up the GitHub repository for project (2).
  • Set up developer environment. Since development will be easier on Linux, I set up a VirtualBox with Linux (Ubuntu) on my Windows machine. I compiled and installed GRASS and installed Jupyter Notebooks in Ubuntu.
  • Opened my first PR (3) which contains binder dependencies.
  1. What do you plan on doing next week?

I've started working on setting up the GRASS repository to run in Binder (3). This week I plan to continue working on that, set up branches for each of the PRs, and begin writing functions for the grass.jupyter library.

  1. Are you blocked on anything?

No, I'm not currently blocked on anything.

Evaluation Status

Weekly reports

Week 1

With Week 1 wrapping up, here is an update on what I worked on this week and where I'm headed next week.

1) What did I get done this week?

  • Finished binder setup for GRASS main repo. PR was approved. There's a new folder in the main GRASS repo that contains the necessary files to launch the repository in binder.
  • Added Binder Button. PR under review. This button will go in the README.md and allow users to launch a Jupyter Lab of the repository in binder. Users can run GRASS (i.e. a compiled version of the main GRASS repo) in Jupyter Notebooks there.
  • Created an example notebook that is linked from the Binder Button. The notebook was copied from here and demonstrates the existing Jupyter/GRASS integration.
  • Wrote draft of GRASS initiation functions for Jupyter (PR).
  • Created Makefile and init.py file for grass.jupyter (PR).
  • Created a Jupyter Notebook where others can view and test the grass.jupyter functions (PR).

2) What do I plan on doing next week? I plan to start working on display functions and familiarizing myself with Folium.

3) Am I blocked on anything? No, I'm not currently blocked on anything.

Week 2

Here is a brief update from Week 2. You can also check out how the new grass.jupyter module works so far in Binder at this link

1) What did I accomplish this week?

  • Had a productive discussion with my mentors: Vaclav Petras, Helena Mitasova and Stefan Blumentrath.
  • Edited and merged Binder button into README PR and GRASS session initiation functions for Jupyter PR.
  • Wrote functions for displaying non-interactive maps in Jupyter and opened PR for discussion PR.
  • Began researching and drafting interactive display options with folium. In particular, figuring out how to best pass GRASS data/layers to folium.

2) What do I plan on doing next week?

  • Edit non-interactive display functions.
  • Continue working on passing GRASS data/layers to folium.

3) Am I blocked on anything? No, I'm not blocked but I didn't get much done this week because I had a family event.

Any suggestions and feedback are welcome!

Week 3

Here is a brief update on what I accomplished during Week 3 of Google Summer of Code for the new GRASS library grass.jupyter. I have also started posting links to Binder Jupyter Notebooks in each PR if you would like to try any of the features and functions I've been working on.

1) What did I accomplish this week?

  • Edited and discussed functions for displaying non-interactive maps in Jupyter PR.
  • Drafted an interactive display function for rasters with folium PR.
  • Made a few minor edits to the example_notebook.ipynb that is linked in the README PR.

2) What do I plan on doing next week?

  • I have a meeting planned with my mentors.
  • I will continue working on interactive functions for Jupyter. In particular, I'm working on passing GRASS rasters to folium.
  • I'll begin writing functions for displaying GRASS vectors with folium.

3) Am I blocked on anything? I'm not blocked at the moment. But, I have been working on finding a good way to pass raster data between GRASS and Jupyter and I'm not sure my current method is particularly fast or robust [3]. Luckily, I have a meeting tomorrow with my mentors where we will discuss this.

Any suggestions and feedback are welcome!

Week 4

With week 4 wrapping up, here is a brief update of what I've been working on. Per usual, you can try proposed changes out in the Binder link in the description of each PR.

1) What did I accomplish during Week 1?

  • I had a productive meeting with my mentors: Vaclav Petras, Helena Mitasova and Stefan Blumentrath. We discussed how environments work and the purpose of copying the environment each time an instance of GrassRenderer is called.
  • I finished and merged a class called GrassRenderer for non-interactive map displays PR.
  • I continued to work on interactive mapping with folium. In particular, I changed how data is passed to folium so that it is first reprojected to WGS84 (required by folium/leaflet) in a new location/mapset PR.

2) What do I plan on doing next week?

  • Continue to work on interactive functions with folium.

3) Am I blocked on anything?

  • No, I'm not blocked on anything at the moment.

Any suggestions and feedback welcome!

Week 5

Here is a weekly update on my project, Improved Integration of GRASS GIS and Jupyter Notebooks.

1) What did I accomplish this week?

  • This week, I spent most of my time improving and editing a new class called InteractiveMap to display vectors interactively with folium. You can test the current functionality in Binder.
  • I had a productive meeting with my mentor, Vaclav Petras on Thursday. We discussed several things that I was a little stuck on and had questions about.
  • Updated grass.jupyter demonstration Notebook.
  • Resolved merge conflicts with master - now, the interactive and non-interactive classes are functional together PR.
  • Found unexpected behavior in the legend display. Calling folium.map.LayerControl() adds a legend to the map but also seems to prevent further modification of the map.

2) What do I plan to do next week?

  • Meet with my mentors tomorrow.
  • GSoC Evaluations.
  • Finish and merge interactive vector methods.
  • Begin interactive maps for rasters.

3) Am I blocked on anything?

  • No, I'm not currently blocked on anything.

Feedback, comments, questions and ideas welcome!

Week 6

Here's a weekly update on my GSoC project, Improved Integration of GRASS GIS and Jupyter Notebooks.

1) What did I accomplish this week?

  • This week, I had two meetings with my mentor, Vaclav Petras. We've been discussing how to improve my workflow, best practices and conventions for GRASS and Python and questions that come up as I'm working.
  • Instituted the use of temporary PNG files for displaying static/non-interactive maps PR and temporary directories for passing vectors between GRASS and folium link (PR coming soon).
  • Continued to edit the InteractiveMap class and mapping GRASS vectors in folium PR
  • Created a shortcut for calling GRASS display modules using getattr PR
  • Completed evaluation and received feedback.

2) What do I plan to do next week?

  • I have another meeting planning with Vaclav tomorrow.
  • Finish and merge existing PRs
  • Continue writing test module for Non-Interactive Vectors
  • Support Raster display in folium map
  • Clean-up function

3) Am I blocked on anything?

  • No, I'm not currently blocked on anything.

Feedback, comments, questions and ideas welcome!

Week 7

Here's a quick update on what I worked on this past week.

1) What did I accomplish this week?

  • This week, I had another two meetings with my mentor, Vaclav Petras. We've been discussing best practices and conventions for GRASS and Python and questions that come up as I'm working.
  • Merged 'add_vector' method for InteractiveMaps in folium [2].
  • Improved error handling and documentation on the shortcut for calling GRASS display modules for "GrassRenderer" [3].
  • Wrote a test module for the "GrassRenderer" class - a class which displays GRASS maps in Jupyter Notebooks as PNG images. I haven't opened a PR since there are a couple things I need to add after some other PRs are merged.
  • Continued to work on overlaying rasters in folium - this is taking me a while because the raster and raster bounds have to be reprojected since folium only takes WGS84 and Pseudo-Mercator projections.

2) What do I plan to do next week?

  • I have another meeting planning with Vaclav on Monday.
  • Finish and merge existing PRs
  • Continue writing test module for GrassRenderer

3) Am I blocked on anything?

  • No, I'm not currently blocked on anything.

Feedback, comments, questions and ideas welcome!

Week 8

Here's a quick update on what I worked on this past week. I came down with a stomach bug this week and didn't get as much as I hoped done but I did wrap up some things I'd been working on for a while.

1) What did I accomplish this week?

  • I had a brief meeting with my mentor, Vaclav Petras. We discussed some of the existing PR's I had open and goals for the rest of the summer.
  • Got rasters to display with folium (finally)! You can check out the functionality in Binder here and see the PR here. It takes a while to reproject the raster (folium only excepts rasters in Pseudo-mercator). As Stefan recommended, I'm hoping to write a "simple CRS" option that won't require reproject and will be faster. But that's for next week...
  • Merged shortcuts and temporary files for GrassRenderer (PR and PR).
  • Continued working on test script for GrassRenderer, the non-interactive display class PR.

2) What do I plan to do next week?

  • I have a meeting with my mentors planned for tomorrow
  • Add simple CRS option for raster display in folium
  • Edit and finish exiting PRs
  • Add option to save/export folium map as HTML

3) Am I blocked on anything?

  • No, I'm not currently blocked on anything.

Feedback, comments, questions and ideas welcome!

Discussion Notes

Future Work

Attachments (2)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.