Opened 4 years ago
Closed 2 years ago
#2247 closed task (fixed)
Python and Jupyter
Reported by: | darkblueb | Owned by: | |
---|---|---|---|
Priority: | normal | Milestone: | OSGeoLive15.0 |
Component: | OSGeoLive | Keywords: | jupyter, python |
Cc: |
Description (last modified by )
ubuntu focal 20.04 LTS LXQt
Install python and jupyter from the default Ubuntu repositories; note the results.
my first goal was to have a packaged setup that satisfies the pySAL mapclassify requirement, which is sensible and they wrote it recently . it is now [ numpy matplotlib descartes scipy sklearn pandas seaborn geopandas ]
https://github.com/pysal/mapclassify
https://github.com/pysal/mapclassify/blob/master/requirements.txt
$ grep Get /home/shared/OSGEO_Misc/o14_misc/py_installs_raw_24jun2020.txt | sed -e 's/^Get.*focal\///' -e 's/ amd64 /\t/' -e 's/\]//' -e 's/ \[/\t/' | sort -u main fonts-font-awesome all 5.0.10+really4.7.0~dfsg-1 514 kB main libamd2 amd64 1:5.7.1+dfsg-2 20.0 kB main libcamd2 amd64 1:5.7.1+dfsg-2 21.7 kB main libccolamd2 amd64 1:5.7.1+dfsg-2 23.2 kB main libcholmod3 amd64 1:5.7.1+dfsg-2 308 kB main libmetis5 amd64 5.1.0.dfsg-5 169 kB main libumfpack5 amd64 1:5.7.1+dfsg-2 229 kB main python3-click all 7.0-3 64.8 kB main python3-colorama all 0.4.3-1build1 23.9 kB main python3-defusedxml all 0.6.0-2 37.0 kB main python3-munch all 2.3.2-2 7,984 B main python3-packaging all 20.3-1 26.8 kB main python3-psutil amd64 5.5.1-1ubuntu4 141 kB main python3-wcwidth all 0.1.8+dfsg1-3 17.4 kB main python3-wrapt amd64 1.11.2-2build1 30.6 kB universe fonts-glyphicons-halflings all 1.009~3.4.1+dfsg-1 117 kB universe ipython3 all 7.13.0-1 4,988 B universe jupyter all 4.6.3-3 3,368 B universe jupyter-client all 6.1.2-1 5,812 B universe jupyter-console all 6.0.0-2 2,820 B universe jupyter-core all 4.6.3-3 5,196 B universe jupyter-nbconvert all 5.6.1-1 4,284 B universe jupyter-nbextension-jupyter-js-widgets all 6.0.0-6 214 kB universe jupyter-nbformat all 5.0.4-1 3,228 B universe jupyter-notebook all 6.0.3-2 5,952 B universe jupyter-qtconsole all 4.6.0-1 5,328 B universe libblosc1 amd64 1.17.1+ds1-1 32.2 kB universe libdsdp-5.8gf amd64 5.8-9.4build1 184 kB universe libglpk40 amd64 4.65-2 378 kB universe libjs-backbone all 1.3.3~dfsg-5 176 kB universe libjs-bootstrap all 3.4.1+dfsg-1 124 kB universe libjs-bootstrap-tour all 0.12.0+dfsg-2 20.4 kB universe libjs-codemirror all 5.51.0-1 609 kB universe libjs-es6-promise all 4.2.8-6 14.7 kB universe libjs-jed all 1.1.1-1 12.2 kB universe libjs-jquery-typeahead all 2.11.0+dfsg1-1 47.0 kB universe libjs-marked all 0.5.1+dfsg-1 17.7 kB universe libjs-moment all 2.24.0+ds-2 126 kB universe libjs-requirejs all 2.3.6-1 29.9 kB universe libjs-requirejs-text all 2.0.12-1 8,894 B universe libjs-text-encoding all 0.7.0-1 104 kB universe libjs-xterm all 3.8.1-3 56.0 kB universe liblbfgsb0 amd64 3.0+dfsg.3-7build1 29.5 kB universe libspatialindex-c6 amd64 1.9.3-1build1 58.3 kB universe libspatialindex-dev amd64 1.9.3-1build1 16.0 kB universe pandoc amd64 2.5-3build2 15.4 MB universe pandoc-data all 2.5-3build2 76.0 kB universe python3-affine all 2.3.0-1 13.8 kB universe python3-atomicwrites all 1.1.5-2build1 7,016 B universe python3-backcall all 0.1.0-2 11.2 kB universe python3-bleach all 3.1.1-1 108 kB universe python3-blosc amd64 1.7.0+ds1-2ubuntu2 22.0 kB universe python3-click-plugins all 1.1.1-2 9,704 B universe python3-cligj all 0.5.0-2 7,708 B universe python3-cloudpickle all 1.3.0-1 23.4 kB universe python3-cvxopt amd64 1.2.3+dfsg-2build1 259 kB universe python3-dask all 2.8.1+dfsg-0.4 532 kB universe python3-deprecated all 1.2.7-1 8,432 B universe python3-descartes all 1.1.0-3 6,108 B universe python3-et-xmlfile all 1.0.1-2.1 9,224 B universe python3-fiona amd64 1.8.13-1build3 469 kB universe python3-fsspec all 0.6.1-0.1 45.7 kB universe python3-geographiclib amd64 1.50.1-1build1 32.2 kB universe python3-geojson all 2.5.0-2 13.3 kB universe python3-geopandas all 0.7.0-1 861 kB universe python3-geopy all 1.20.0-1 53.4 kB universe python3-ipykernel all 5.2.0-1 76.8 kB universe python3-ipython all 7.13.0-1 494 kB universe python3-ipywidgets all 6.0.0-6 31.0 kB universe python3-jdcal all 1.0-1.3 7,944 B universe python3-jedi all 0.15.2-1 502 kB universe python3-joblib all 0.14.0-3 192 kB universe python3-jupyter-client all 6.1.2-1 69.5 kB universe python3-jupyter-console all 6.0.0-2 18.2 kB universe python3-locket all 0.2.0-1 4,562 B universe python3-mistune all 0.8.4-2 14.3 kB universe python3-nbconvert all 5.6.1-1 102 kB universe python3-nose all 1.3.7-5 116 kB universe python3-notebook all 6.0.3-2 1,094 kB universe python3-numexpr amd64 2.7.1-1build1 133 kB universe python3-openpyxl all 3.0.3-1 144 kB universe python3-pandas all 0.25.3+dfsg-7 1,968 kB universe python3-pandas-lib amd64 0.25.3+dfsg-7 4,628 kB universe python3-pandocfilters all 1.4.2-2 18.9 kB universe python3-parso all 0.5.2-1ubuntu1 62.8 kB universe python3-partd all 1.0.0-1 14.8 kB universe python3-patsy all 0.5.1-1 168 kB universe python3-pickleshare all 0.7.5-2 7,560 B universe python3-pluggy all 0.13.0-2 18.4 kB universe python3-prometheus-client all 0.7.1-1.1 33.3 kB universe python3-prompt-toolkit all 2.0.10-2 220 kB universe python3-py all 1.8.1-1 65.4 kB universe python3-pysal all 2.1.0-1 14.3 MB universe python3-pytest all 4.6.9-1 178 kB universe python3-qtconsole all 4.6.0-1 87.6 kB universe python3-rasterio amd64 1.1.3-1 997 kB universe python3-rtree all 0.9.4-1 54.0 kB universe python3-scipy amd64 1.3.3-3build1 11.1 MB universe python3-seaborn all 0.10.0-1 155 kB universe python3-send2trash all 1.5.0-2 9,704 B universe python3-shapely amd64 1.7.0-1build1 248 kB universe python3-sklearn all 0.22.2.post1+dfsg-5 1,773 kB universe python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 2,358 kB universe python3-snuggs all 1.4.7-1 6,744 B universe python3-statsmodels all 0.11.1-1 3,939 kB universe python3-statsmodels-lib amd64 0.11.1-1 1,574 kB universe python3-tables all 3.6.1-2build1 333 kB universe python3-tables-lib amd64 3.6.1-2build1 501 kB universe python3-terminado all 0.8.2-3 9,312 B universe python3-testpath all 0.4.4+dfsg-1 9,040 B universe python3-toolz all 0.9.0-1 42.0 kB universe python3-tornado amd64 6.0.3+really5.1.1-3 304 kB universe python3-tqdm all 4.30.0-1 68.4 kB universe python3-widgetsnbextension all 6.0.0-6 4,888 B universe python3-xlrd all 1.1.0-5 90.8 kB universe python3-xlwt all 1.3.0-3 83.7 kB universe python3-zmq amd64 18.1.1-3 294 kB universe python-tables-data all 3.6.1-2build1 45.7 kB universe rasterio all 1.1.3-1 8,868 B
Change History (14)
comment:1 by , 4 years ago
Description: | modified (diff) |
---|
comment:2 by , 4 years ago
comment:3 by , 4 years ago
build ol14b187
$PATH: /home/user/bin /usr/local/sbin /usr/local/bin /usr/sbin /usr/bin /sbin /bin /usr/games /usr/local/games /snap/bin /usr/lib/gmt/bin sys.path: /usr/bin /usr/lib/python38.zip /usr/lib/python3.8 /usr/lib/python3.8/lib-dynload /home/user/.local/lib/python3.8/site-packages /usr/local/lib/python3.8/dist-packages /usr/lib/python3/dist-packages sys.executable: /usr/bin/python3 sys.version: 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0] platform.platform(): Linux-5.4.0-42-generic-x86_64-with-glibc2.29 which -a jupyter: /usr/bin/jupyter /bin/jupyter
comment:4 by , 4 years ago
stable, first pass at python3 installs using Ubuntu packaging
gdal numpy rasterio fiona shapely matplotlib descartes seaborn dask owslib psycopg2 sqlalchemy sqlalchemy-ext geoalchemy2 statsmodels statsmodels-lib pandas pandas-lib geopandas geojson pysal rtree geographiclib scipy sklearn sklearn-lib geopy jupyter-client jupyter-console jupyter-core nbconvert nbformat notebook ipykernel ipython-genutils ipython3 ipywidgets qgis qgis-common affine apport apt aptdaemon atomicwrites attr backcall bleach blinker blosc bs4 certifi cffi-backend chardet click click-plugins cligj cloudpickle colorama commandnotfound cryptography cups cupshelpers cvxopt cycler dateutil dbus debconf debian decorator defer defusedxml deprecated dev distro distro-info distupgrade distutils entrypoints examples fsspec future gi html5lib httplib2 idna importlib-metadata jdcal jedi jinja2 joblib jsonschema jwt keyring kiwisolver launchpadlib lib2to3 locket lxml markupsafe minimal mistune more-itertools munch nacl netifaces nose numexpr oauthlib olefile opengl openpyxl packaging pandocfilters parso partd patsy pexpect pickleshare pil pip pkg-resources plotly pluggy problem-report prometheus-client prompt-toolkit psutil ptyprocess pygments py pymacaroons pyparsing pyproj pyqt5 pyqt5.qsci pyqt5.qtsql pyqt5.qtsvg pyqt5.qtwebkit pyrsistent pytest qtconsole renderpm reportlab-accel reportlab requests requests-unixsocket retrying secretstorage send2trash setuptools simplejson sip six snuggs software-properties soupsieve systemd tables tables-lib terminado testpath toolz tornado tqdm traitlets tz uno update-manager urllib3 wadllib wcwidth webencodings wheel widgetsnbextension wrapt xapian xdg xkit xlrd xlwt yaml zipp zmq
comment:5 by , 4 years ago
python3-dask
Package: python3-dask Version: 2.8.1+dfsg-0.4 Section: universe/python - Recommends: python3-cloudpickle, python3-numpy, python3-pandas, python3-partd, python3-requests Suggests: ipython, python-dask-doc, python3-bcolz, python3-blosc, python3-boto, python3-distributed (>= 1.21), python3-graphviz, python3-h5py, python3-psutil, python3-scipy, python3-skimage, python3-sklearn, python3-sqlalchemy, python3-tables Homepage: https://github.com/dask/dask
comment:6 by , 4 years ago
Jupyter Introduction
Pangeo-Data Repo https://github.com/pangeo-data/helm-chart
Pangeo Notebooks https://github.com/pangeo-data/pangeo-example-notebooks
comment:7 by , 4 years ago
Ubuntu/Debian/GNU Linux System Python v3.8 Linux-5.4.0-42-generic-x86_64-with-glibc2.29 SciPy SkLearn Matplotlib installed via system .deb # -- import mapclassify ------------------------------ import 'platform' # import 'multiprocessing' # import 'numpy' # 1.17.4 import 'scipy.spatial.kdtree' # 1.3.3 import 'scipy.spatial.ckdtree' # import 'scipy.spatial.qhull' # import 'scipy.spatial._voronoi' # import 'scipy.spatial._distance_wrap' # import 'scipy.spatial._hausdorff' # import 'scipy.spatial.distance' # import 'scipy.spatial._spherical_voronoi' # import 'scipy.spatial.transform.rotation' # import 'scipy.spatial.transform._rotation_spline' # import 'scipy.spatial.transform' # import 'scipy.spatial' # import 'scipy.optimize._shgo_lib.triangulation' # import 'scipy.optimize._dual_annealing' # import 'scipy.optimize' # import 'scipy.stats' # import 'sklearn._config' # import 'sklearn._distributor_init' # import 'sysconfig' # import 'json' # import 'zipfile' # import 'xml' # import 'xml.parsers.expat' # import 'joblib' # import 'sklearn.utils._joblib' # import 'sklearn.exceptions' # import 'distutils' # import 'sklearn.externals' # import 'sklearn.neighbors._dist_metrics' # import 'sklearn.neighbors._ball_tree' # import 'sklearn.neighbors._kd_tree' # import 'sklearn.neighbors._base' # import 'sklearn.neighbors._unsupervised' # import 'sklearn.neighbors._graph' # import 'sklearn.neighbors._classification' # import 'sklearn.neighbors._regression' # import 'sklearn.neighbors._nearest_centroid' # import 'sklearn.neighbors._kde' # import 'sklearn.neighbors._lof' # import 'sklearn.tree._utils' # import 'sklearn.tree._tree' # import 'sklearn.tree._splitter' # import 'sklearn.tree._criterion' # import 'sklearn.tree._classes' # import 'sklearn.tree._reingold_tilford' # import 'sklearn.tree._export' # import 'sklearn.tree' # import 'sklearn.neighbors._quad_tree' # import 'sklearn.manifold' # import 'sklearn.cluster._k_means_fast' # import 'sklearn.cluster._k_means_elkan' # import 'sklearn.cluster._kmeans' # import 'sklearn.cluster._spectral' # import 'sklearn.cluster._mean_shift' # import 'sklearn.cluster._affinity_propagation' # import 'sklearn.utils._fast_dict' # import 'sklearn.cluster._hierarchical_fast' # import 'sklearn.cluster._feature_agglomeration' # import 'sklearn.cluster._agglomerative' # import 'sklearn.cluster._dbscan_inner' # import 'sklearn.cluster._dbscan' # import 'sklearn.cluster._optics' # import 'sklearn.cluster._bicluster' # import 'sklearn.cluster._birch' # import 'sklearn.cluster' # 0.22.2.post1 import 'mapclassify.classifiers' # import 'mapclassify.pooling' # import 'mapclassify.greedy' # import 'mapclassify' # 2.3.0 ## pandas shapely installed by system .deb ##-- import geopandas -------------------------------- import 'six' # import 'pandas._config.config' # 0.25.3 import 'pandas._libs' # import 'pandas.core' # import 'pandas.io' # import 'pandas.io.formats' # import 'pandas.io.formats.console' # import 'unicodedata' # import 'csv' # import 'gzip' # import 'http' # import 'pandas.io.common' # import 'pandas.io.formats.format' # import 'matplotlib' # 3.1.2 import 'matplotlib.cm' # import 'pandas.plotting._matplotlib.style' # import 'matplotlib.font_manager' # import 'pandas.api' # import 'pandas.io.spss' # import 'pandas.io.sql' # import 'pandas.io.stata' # import 'pandas.io.api' # import 'pandas' # 0.25.3 import 'shapely' # 1.7.0 import 'rtree' # import 'pyproj.compat' # import 'pyproj' # 2.5.0 import 'shapely.geos' # libgeos-3.8.0 import 'shapely.speedups._speedups' # import 'shapely.prepared' # import 'geopandas.io' # import 'geopandas.geodataframe' # import 'fiona' # import 'geopandas.io.file' # import 'geopandas.io.arrow' # import 'geopandas.io.sql' # import 'geopandas._version' # import 'geopandas' # 0.8.0+5.g89908f8
comment:8 by , 4 years ago
TODO: consider geoplot https://github.com/ResidentMario/geoplot API https://residentmario.github.io/geoplot/api_reference.html
comment:9 by , 4 years ago
STAC support August 2020
Tutorial for pySTAC https://pystac.readthedocs.io/en/latest/tutorials/pystac-spacenet-tutorial.html
https://github.com/stac-utils
note: boto3
and s3fs
are supplied as dot-deb, but specific to Amazon S3
compare & contrast
comment:10 by , 4 years ago
folium install via pip3
user@ol14b187:~$ pip3 install --user --upgrade-strategy=only-if-needed folium Collecting folium Downloading folium-0.11.0-py2.py3-none-any.whl (93 kB) |████████████████████████████████| 93 kB 247 kB/s Requirement already satisfied: branca>=0.3.0 in /usr/lib/python3/dist-packages (from folium) (0.3.1) Requirement already satisfied: numpy in /usr/lib/python3/dist-packages (from folium) (1.17.4) Requirement already satisfied: jinja2>=2.9 in /usr/lib/python3/dist-packages (from folium) (2.10.1) Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from folium) (2.22.0) Installing collected packages: folium Successfully installed folium-0.11.0
comment:11 by , 4 years ago
joris and crew use their own maptile utility -- contextily
user@ol14b187:$ pip3 install --user contextily Collecting contextily Downloading contextily-1.0.1-py3-none-any.whl (23 kB) Requirement already satisfied: pillow in /usr/lib/python3/dist-packages (from contextily) (7.0.0) Collecting geopy Downloading geopy-2.0.0-py3-none-any.whl (111 kB) |████████████████████████████████| 111 kB 381 kB/s Requirement already satisfied: rasterio in /usr/lib/python3/dist-packages (from contextily) (1.1.8) Requirement already satisfied: joblib in /usr/lib/python3/dist-packages (from contextily) (0.14.0) Requirement already satisfied: matplotlib in /usr/lib/python3/dist-packages (from contextily) (3.1.2) Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from contextily) (2.22.0) Requirement already satisfied: mercantile in /usr/lib/python3/dist-packages (from contextily) (1.1.6) Requirement already satisfied: geographiclib<2,>=1.49 in /usr/lib/python3/dist-packages (from geopy->contextily) (1.50) Installing collected packages: geopy, contextily Successfully installed contextily-1.0.1 geopy-2.0.0
comment:12 by , 4 years ago
Jupyter info on build ol14b187
user@ol14b187:~/jupyter$ jupyter --version jupyter core : 4.6.3 jupyter-notebook : 6.0.3 qtconsole : not installed ipython : 7.13.0 ipykernel : 5.2.0 jupyter client : 6.1.2 jupyter lab : not installed nbconvert : 5.6.1 ipywidgets : 6.0.0 nbformat : 5.0.4 traitlets : 4.3.3
comment:13 by , 3 years ago
Milestone: | OSGeoLive14.0 → OSGeoLive15.0 |
---|
Ticket retargeted after milestone closed
hmm regarding
pandas