#2150 closed defect (fixed)
actinia startup error
Reported by: | neteler | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | OSGeoLive13.0 |
Component: | OSGeoLive | Keywords: | actinia |
Cc: | osgeolive@… |
Description
I have prepared an installer for "actinia_core" (a new OSGeo community project, https://www.osgeo.org/projects/actinia/) which we will present at FOSS4G 2019, Bucharest.
Pull request of installer script: https://github.com/OSGeo/OSGeoLive/pull/266
Background:
Actinia is an open source REST API for scalable, distributed, high performance processing of geographical data that uses GRASS GIS for computational tasks. It provides a REST API to process satellite images, time series of satellite images, arbitrary raster data with geographical relations and vector data.
Change history (17)
comment:1 Changed 19 months ago by
comment:2 Changed 18 months ago by
Actinia debian package now available in out ppa.
The installer needs a review.
comment:3 Changed 18 months ago by
Cc: | osgeolive@… added |
---|---|
Priority: | normal → critical |
comment:4 Changed 18 months ago by
Actinia debian package needs more dependencies to work properly:
python3-google-api-core, python3-google-cloud-core, python3-google-auth, python3-google-cloud-bigquery, python3-google-cloud-storage, python3-rq
comment:5 Changed 18 months ago by
Priority: | critical → major |
---|---|
Summary: | New installer for actinia → actinia missing dependencies |
Type: | enhancement → defect |
comment:6 Changed 18 months ago by
In my view there is no need to include these dependencies since Google Cloud Processing requires an API key which isn't to be included in OSGeoLive
comment:7 Changed 18 months ago by
user@osgeolive:~$ actinia_start.sh [2019-08-01 17:59:42 +0000] [2933] [INFO] Starting gunicorn 19.7.1 [2019-08-01 17:59:42 +0000] [2933] [INFO] Listening at: http://0.0.0.0:8088 (2933) [2019-08-01 17:59:42 +0000] [2933] [INFO] Using worker: sync [2019-08-01 17:59:42 +0000] [2936] [INFO] Booting worker with pid: 2936 [2019-08-01 17:59:42 +0000] [2936] [ERROR] Exception in worker process Traceback (most recent call last): File "/usr/lib/python3/dist-packages/gunicorn/arbiter.py", line 578, in spawn_worker worker.init_process() File "/usr/lib/python3/dist-packages/gunicorn/workers/base.py", line 126, in init_process self.load_wsgi() File "/usr/lib/python3/dist-packages/gunicorn/workers/base.py", line 135, in load_wsgi self.wsgi = self.app.wsgi() File "/usr/lib/python3/dist-packages/gunicorn/app/base.py", line 67, in wsgi self.callable = self.load() File "/usr/lib/python3/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load return self.load_wsgiapp() File "/usr/lib/python3/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp return util.import_app(self.app_uri) File "/usr/lib/python3/dist-packages/gunicorn/util.py", line 377, in import_app __import__(module) File "/usr/lib/python3/dist-packages/actinia_core/main.py", line 30, in <module> from .endpoints import create_endpoints File "/usr/lib/python3/dist-packages/actinia_core/endpoints.py", line 33, in <module> from .resources.location_management import ListLocationsResource, LocationManagementResourceUser File "/usr/lib/python3/dist-packages/actinia_core/resources/location_management.py", line 39, in <module> from .common.response_models import ProcessingResponseModel File "/usr/lib/python3/dist-packages/actinia_core/resources/common/response_models.py", line 33, in <module> from .process_chain import GrassModule File "/usr/lib/python3/dist-packages/actinia_core/resources/common/process_chain.py", line 35, in <module> from .sentinel_processing_library import Sentinel2Processing File "/usr/lib/python3/dist-packages/actinia_core/resources/common/sentinel_processing_library.py", line 30, in <module> from .google_satellite_bigquery_interface import GoogleSatelliteBigQueryInterface File "/usr/lib/python3/dist-packages/actinia_core/resources/common/google_satellite_bigquery_interface.py", line 28, in <module> from google.cloud import bigquery File "/usr/lib/python3/dist-packages/google/cloud/bigquery/__init__.py", line 35, in <module> from google.cloud.bigquery.client import Client File "/usr/lib/python3/dist-packages/google/cloud/bigquery/client.py", line 52, in <module> from google.cloud.bigquery import _pandas_helpers File "/usr/lib/python3/dist-packages/google/cloud/bigquery/_pandas_helpers.py", line 24, in <module> from google.cloud import bigquery_storage_v1beta1 File "/usr/lib/python3/dist-packages/google/cloud/bigquery_storage_v1beta1/__init__.py", line 25, in <module> from google.cloud.bigquery_storage_v1beta1 import types File "/usr/lib/python3/dist-packages/google/cloud/bigquery_storage_v1beta1/types.py", line 23, in <module> from google.cloud.bigquery_storage_v1beta1.proto import arrow_pb2 File "/usr/lib/python3/dist-packages/google/cloud/bigquery_storage_v1beta1/proto/arrow_pb2.py", line 26, in <module> '\n7google/cloud/bigquery/storage_v1beta1/proto/arrow.proto\x12%google.cloud.bigquery.storage.v1beta1"(\n\x0b\x41rrowSchema\x12\x19\n\x11serialized_schema\x18\x01 \x01(\x0c"F\n\x10\x41rrowRecordBatch\x12\x1f\n\x17serialized_record_batch\x18\x01 \x01(\x0c\x12\x11\n\trow_count\x18\x02 \x01(\x03\x42\x85\x01\n)com.google.cloud.bigquery.storage.v1beta1B\nArrowProtoZLgoogle.golang.org/genproto/googleapis/cloud/bigquery/storage/v1beta1;storageb\x06proto3' TypeError: __new__() got an unexpected keyword argument 'serialized_options' [2019-08-01 17:59:42 +0000] [2936] [INFO] Worker exiting (pid: 2936) [2019-08-01 17:59:42 +0000] [2933] [INFO] Shutting down: Master [2019-08-01 17:59:42 +0000] [2933] [INFO] Reason: Worker failed to boot.
comment:8 Changed 18 months ago by
Without the google dependencies, there are several import errors on startup. Is there a way to drop these google dependencies by configuration?
comment:9 Changed 18 months ago by
The dependencies cannot be dropped without major refactoring (may come in future).
Probably Google Cloud API version differences are the reason:
https://github.com/mundialis/actinia_core/blob/master/requirements.txt
- google-cloud==0.32.0
- google-cloud-bigquery==0.28.0
- google-cloud-storage==1.6.0
(known to work)
vs https://launchpad.net/~gcpp-kalxas/+archive/ubuntu/actinia
- google-cloud 0.32.0-0~bionic1 (the same)
- google-cloud-bigquery 1.17.0-0~bionic0 - different
- google-cloud-bigquery-storage 0.6.0-0~bionic0
- google-cloud-core 1.0.3-0~bionic1
- google-cloud-storage 1.17.0-0~bionic0 - different
comment:10 Changed 18 months ago by
These are the requirements of google-cloud 0.32 according to the setup.py here:
REQUIREMENTS = [ 'google-api-core >= 0.1.2, < 0.2.0dev', 'google-cloud-bigquery >= 0.28.0, < 0.29dev', 'google-cloud-bigquery-datatransfer >= 0.1.0, < 0.2dev', 'google-cloud-bigtable >= 0.28.1, < 0.29dev', 'google-cloud-container >= 0.1.0, < 0.2dev', 'google-cloud-core >= 0.28.0, < 0.29dev', 'google-cloud-datastore >= 1.4.0, < 1.5dev', 'google-cloud-dns >= 0.28.0, < 0.29dev', 'google-cloud-error-reporting >= 0.28.0, < 0.29dev', 'google-cloud-firestore >= 0.28.0, < 0.29dev', 'google-cloud-language >= 1.0.0, < 1.1dev', 'google-cloud-logging >= 1.4.0, < 1.5dev', 'google-cloud-monitoring >= 0.28.0, < 0.29dev', 'google-cloud-pubsub >= 0.30.0, < 0.31dev', 'google-cloud-resource-manager >= 0.28.0, < 0.29dev', 'google-cloud-runtimeconfig >= 0.28.0, < 0.29dev', 'google-cloud-spanner >= 0.29.0, < 0.30dev', 'google-cloud-speech >= 0.30.0, < 0.31dev', 'google-cloud-storage >= 1.6.0, < 1.7dev', 'google-cloud-trace >= 0.17.0, < 0.18dev', 'google-cloud-translate >= 1.3.0, < 1.4dev', 'google-cloud-videointelligence >= 1.0.0, < 1.1dev', 'google-cloud-vision >= 0.29.0, < 0.30dev', ]
comment:11 Changed 18 months ago by
Fixed versions in https://launchpad.net/~gcpp-kalxas/+archive/ubuntu/osgeolive/+packages
Lets see if that will do it.
comment:12 Changed 18 months ago by
user@osgeolive:/opt$ actinia_start.sh [2019-08-05 08:45:16 +0000] [7710] [INFO] Starting gunicorn 19.7.1 [2019-08-05 08:45:16 +0000] [7710] [INFO] Listening at: http://0.0.0.0:8088 (7710) [2019-08-05 08:45:16 +0000] [7710] [INFO] Using worker: sync [2019-08-05 08:45:16 +0000] [7713] [INFO] Booting worker with pid: 7713 from actinia_statistic_plugin.endpoints import create_endpoints as create_plugin_endpoints create_plugin_endpoints(flask_api=flask_api) ERROR: Could not connect to redis with localhost 6379 pass Client sent AUTH, but no password is set ERROR: Could not connect to redis with localhost 6379 pass Client sent AUTH, but no password is set ERROR: Could not connect to redis with localhost 6379 pass Client sent AUTH, but no password is set
comment:13 Changed 18 months ago by
Summary: | actinia missing dependencies → actinia startup error |
---|
comment:14 Changed 18 months ago by
An idea from (via huhabla):
Can you check if the REDIS_URL environment variable is set? If that's the case, the used Redis client may be picking the password from it.
comment:15 Changed 18 months ago by
comment:16 follow-up: 17 Changed 18 months ago by
Resolution: | → fixed |
---|---|
Status: | new → closed |
Fixed in build228
Initial docs (actinia overview and quickstart) added as well:
https://github.com/OSGeo/OSGeoLive-doc/pull/455