Context Navigation

← Previous Ticket
Next Ticket →

#2360 closed defect (fixed)

EOxServer fails to launch

Reported by:	darkblueb	Owned by:	osgeolive@…
Priority:	critical	Milestone:	OSGeoLive15.0
Component:	OSGeoLive	Keywords:	eoxserver, django
Cc:	osgeolive@…

Description

.. from apache2 error log

[Sat Jun 11 23:01:33.315768 2022] [wsgi:error] [pid 1239] [remote 127.0.0.1:46524]     from eoxserver.resources.coverages import models as coverage_models
[Sat Jun 11 23:01:33.315773 2022] [wsgi:error] [pid 1239] [remote 127.0.0.1:46524]   File "/usr/lib/python3/dist-packages/eoxserver/resources/coverages/models.py", line 45, in <module>
[Sat Jun 11 23:01:33.315777 2022] [wsgi:error] [pid 1239] [remote 127.0.0.1:46524]     from django.utils.encoding import python_2_unicode_compatible
[Sat Jun 11 23:01:33.315792 2022] [wsgi:error] [pid 1239] [remote 127.0.0.1:46524] ImportError: cannot import name 'python_2_unicode_compatible' from 'django.utils.encoding' (/usr/lib/python3/dist-packages/django/utils/encoding.py)

Attachments (1)

eoxserver-error-log-on-docker.txt (5.7 KB ) - added by sanak 2 years ago.

Download all attachments as: .zip

Change History (15)

comment:1 by kalxas, 2 years ago

Cc:	osgeolive@… added
Summary:	eox server fails to launch → EOxServer fails to launch

comment:2 by sanak, 2 years ago

From googling a bit, Django 3 seems to drop the python_2_unicode_compatible method. https://docs.djangoproject.com/en/3.0/releases/3.0/#removed-private-python-2-compatibility-apis

And EOxServer seems to limit django<3 in the latest 1.1.2.

https://github.com/EOxServer/eoxserver/commit/fba879a32d8433698a0c5c3d5d89b308affde32e

So, I guess that sticking Django < 3 (2.2.28 seems to be the latest) may be a workaround for this issue. https://docs.djangoproject.com/en/4.0/releases/#id4

Other Django dependency's GeoNode also seems to need Django 2.2.24+ in 3.3.x branch. https://github.com/GeoNode/geonode/blob/3.3.x/setup.cfg#L33

comment:3 by kalxas, 2 years ago

https://launchpad.net/~gcpp-kalxas/+archive/ubuntu/osgeolive/+sourcepub/13770380/+listing-archive-extra

comment:4 by kalxas, 2 years ago

After downgrading to Django 2.2.24 the next error is this:

Creating EOxServer demonstration instance
/usr/bin/eoxserver-admin.py:35: DeprecationWarning: 'eoxserver-admin.py' is deprecated. Use 'eoxserver-instance.py' to create an instance
  warnings.warn(
Initializing django project folder.
Traceback (most recent call last):
  File "/usr/local/share/eoxserver/eoxserver_demonstration/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/usr/lib/python3/dist-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python3/dist-packages/django/apps/registry.py", line 114, in populate
    app_config.import_models()
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 211, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/eoxserver/services/models.py", line 30, in <module>
    from eoxserver.resources.coverages import models as coverage_models
  File "/usr/lib/python3/dist-packages/eoxserver/resources/coverages/models.py", line 50, in <module>
    from eoxserver.render.browse.generate import (
  File "/usr/lib/python3/dist-packages/eoxserver/render/browse/generate.py", line 112, in <module>
    _ast.Index,
AttributeError: module '_ast' has no attribute 'Index'

comment:5 by sanak, 2 years ago

Okay...

I could reproduce above _ast error and other is_timeout error also on eoxserver docker environment by changing docker base image. https://github.com/EOxServer/eoxserver/blob/master/docker/Readme.md

git clone git@github.com:EOxServer/eoxserver.git
git checkout release-1.1.0
cd eoxserver/
vi ./docker/ubuntu/py3/Dockerfile
# change "FROM ubuntu:20.04" to "FROM ubuntu:22.04"
# and "ARG DJANGO=2.2.17" to "ARG DJANGO=2.2.24", then save
docker build -t eoxserver -f ./docker/ubuntu/py3/Dockerfile .
docker image ls
docker run -i -t --rm -p 8080:8000 <IMAGE ID>

On eoxserver master branch (=1.1.2), the _ast error doesn't happen, but is_timeout error still happens, and to solve this error, the following requirements.txt modification seems to be necessary.

-gunicorn
-eventlet==0.30.2
+https://github.com/benoitc/gunicorn/archive/refs/heads/master.zip#egg=gunicorn==20.1.0
+eventlet==0.33.1

Here are some related links.

by sanak, 2 years ago

Attachment:	eoxserver-error-log-on-docker.txt added

comment:6 by darkblueb, 2 years ago

build 93 shows:

Wed Jun 22 22:25:43.891411 2022] [wsgi:error] [pid 1248] /usr/lib/python3/dist-packages/django/contrib/gis/shortcuts.py:10: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode.
[Wed Jun 22 22:25:43.891458 2022] [wsgi:error] [pid 1248] Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve.
[Wed Jun 22 22:25:43.891462 2022] [wsgi:error] [pid 1248]   import numpy
[Wed Jun 22 22:25:43.954763 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338] mod_wsgi (pid=1248): Failed to exec Python script file '/usr/local/share/eoxserver/eoxserver_demonstration/eoxserver_demonstration/wsgi.py'.
[Wed Jun 22 22:25:43.954820 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338] mod_wsgi (pid=1248): Exception occurred processing WSGI script '/usr/local/share/eoxserver/eoxserver_demonstration/eoxserver_demonstration/wsgi.py'.
[Wed Jun 22 22:25:43.960132 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338] Traceback (most recent call last):
[Wed Jun 22 22:25:43.960180 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "/usr/local/share/eoxserver/eoxserver_demonstration/eoxserver_demonstration/wsgi.py", line 67, in <module>
[Wed Jun 22 22:25:43.960184 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]     application = get_wsgi_application()
[Wed Jun 22 22:25:43.960188 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "/usr/lib/python3/dist-packages/django/core/wsgi.py", line 12, in get_wsgi_application
[Wed Jun 22 22:25:43.960191 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]     django.setup(set_prefix=False)
[Wed Jun 22 22:25:43.960195 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "/usr/lib/python3/dist-packages/django/__init__.py", line 24, in setup
[Wed Jun 22 22:25:43.960198 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]     apps.populate(settings.INSTALLED_APPS)
[Wed Jun 22 22:25:43.960201 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "/usr/lib/python3/dist-packages/django/apps/registry.py", line 114, in populate
[Wed Jun 22 22:25:43.960203 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]     app_config.import_models()
[Wed Jun 22 22:25:43.960207 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "/usr/lib/python3/dist-packages/django/apps/config.py", line 301, in import_models
[Wed Jun 22 22:25:43.960209 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]     self.models_module = import_module(models_module_name)
[Wed Jun 22 22:25:43.960218 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
[Wed Jun 22 22:25:43.960221 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]     return _bootstrap._gcd_import(name[level:], package, level)
[Wed Jun 22 22:25:43.960224 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
[Wed Jun 22 22:25:43.960228 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
[Wed Jun 22 22:25:43.960231 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
[Wed Jun 22 22:25:43.960235 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
[Wed Jun 22 22:25:43.960238 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
[Wed Jun 22 22:25:43.960241 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
[Wed Jun 22 22:25:43.960248 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "/usr/lib/python3/dist-packages/eoxserver/services/models.py", line 30, in <module>
[Wed Jun 22 22:25:43.960251 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]     from eoxserver.resources.coverages import models as coverage_models
[Wed Jun 22 22:25:43.960255 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]   File "/usr/lib/python3/dist-packages/eoxserver/resources/coverages/models.py", line 45, in <module>
[Wed Jun 22 22:25:43.960257 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338]     from django.utils.encoding import python_2_unicode_compatible
[Wed Jun 22 22:25:43.960270 2022] [wsgi:error] [pid 1248] [remote 127.0.0.1:40338] ImportError: cannot import name 'python_2_unicode_compatible' from 'django.utils.encoding' (/usr/lib/python3/dist-packages/django/utils/encoding.py)

comment:7 by kalxas, 2 years ago

django 2.x debian package is on staging ppa so not included in nightly builds. It is included in staging builds only

comment:8 by kalxas, 2 years ago

After upgrade to eoxserver 1.1.2 things are much better.

The latest issue now is this:

CREATE EXTENSION
The user `user' is already a member of `users'.
Creating EOxServer demonstration instance
/usr/bin/eoxserver-admin.py:35: DeprecationWarning: 'eoxserver-admin.py' is deprecated. Use 'eoxserver-instance.py' to create an instance
  warnings.warn(
Initializing django project folder.
Traceback (most recent call last):
  File "/usr/local/share/eoxserver/eoxserver_demonstration/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/usr/lib/python3/dist-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python3/dist-packages/django/apps/registry.py", line 114, in populate
    app_config.import_models()
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 211, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/eoxserver/services/models.py", line 30, in <module>
    from eoxserver.resources.coverages import models as coverage_models
  File "/usr/lib/python3/dist-packages/eoxserver/resources/coverages/models.py", line 47, in <module>
    from jsonfield import JSONField
ModuleNotFoundError: No module named 'jsonfield'

comment:9 by kalxas, 2 years ago

python3-jsonfield is also a dependency for GeoNode, so it will be added to the ppa.

comment:10 by kalxas, 2 years ago

After adding jsonfield, there is progress in the installation but there is a new error when adding data to the database:

Extracting demonstration data in /usr/local/share/eoxserver/eoxserver_demonstration.
INFO: Successfully imported coverage type 'RGB'
Successfully created product type 'MERIS_product_type'
Successfully created collection type 'MERIS_collection_type'
Successfully created collection 'MER_FRS_1P_RGB_reduced'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.NotNullViolation: null value in column "identifier" of relation "coverages_eoobject" violates not-null constraint
DETAIL:  Failing row contains (2, null, null, null, null, 2022-07-03 15:48:36.568182+00, 2022-07-03 15:48:36.568196+00).


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/share/eoxserver/eoxserver_demonstration/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/usr/lib/python3/dist-packages/eoxserver/resources/coverages/management/commands/product.py", line 239, in handle
    self.handle_register(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/eoxserver/resources/coverages/management/commands/product.py", line 264, in handle_register
    product, replaced = ProductRegistrator().register(
  File "/usr/lib/python3/dist-packages/eoxserver/resources/coverages/registration/product.py", line 147, in register
    product = models.Product.objects.create(
  File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 422, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 743, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 779, in save_base
    parent_inserted = self._save_parents(cls, using, update_fields)
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 808, in _save_parents
    updated = self._save_table(
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 873, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 910, in _do_insert
    return manager._insert([self], fields=fields, return_id=update_pk,
  File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 1186, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 80, in _execute
    with self.db.wrap_database_errors:
  File "/usr/lib/python3/dist-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: null value in column "identifier" of relation "coverages_eoobject" violates not-null constraint
DETAIL:  Failing row contains (2, null, null, null, null, 2022-07-03 15:48:36.568182+00, 2022-07-03 15:48:36.568196+00).

follow-up: 12 comment:11 by kalxas, 2 years ago

Upstream fix:

https://github.com/EOxServer/eoxserver/pull/527/commits/513a1e6144ab4b7e2b54a79744cb00b3873782e7

in reply to: 11 comment:12 by sanak, 2 years ago

Replying to kalxas:

Upstream fix:

https://github.com/EOxServer/eoxserver/pull/527/commits/513a1e6144ab4b7e2b54a79744cb00b3873782e7

Thanks !
I confirmed that comment:10 issue was solved from nightly:144 log.

By the way, I still encounter 500 error when staring EOxServer from the menu and /var/log/apache2/error.log recorded the following error log.

Failed to exec Python script file '/usr/local/share/eoxserver/eoxserver_demonstration/eoxserver_demonstration/wsgi.py'.
Exeception occurred processing WSGI script '/usr/local/share/eoxserver/eoxserver_demonstration/eoxserver_demonstration/wsgi.py'.
Traceback (most recent call last):
  File "/usr/local/share/eoxserver/eoxserver_demonstration/eoxserver_demonstration/wsgi.py", line 60, in <module>
    import eoxserver.core
ModuleNotFoundError: No module named 'eoxserver'

Actually, python3-eoxserver package is missing, and after installing python3-eoxserver along with python3-zipstream, the EOxServer service launched correctly.

$ dpkg -l | grep python3-eoxserver
# Nothing
$ sudo apt install python3-eoxserver
:
The following packages have unmet dependencies:
 python3-zipstream-ng : Breaks : python3-zipstream but 1.1.4-1 is to be installed
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
$ sudo apt install python3-zipstream python3-eoxserver
:
The following packages will be REMOVED:
  python3-geonode python3-zipstream-ng
:

I guess that python3-eoxserver and python3-geonode may conflict each other.

Version 0, edited 2 years ago by sanak (next)

comment:13 by kalxas, 2 years ago

Good catch @sanak

Indeed python3-zipstream and python3-zipstream-ng are conflicting by Ubuntu packaging.

I am going to test if both eoxserver and geonode can work with the same library.

comment:14 by kalxas, 2 years ago

Resolution:	→ fixed
Status:	new → closed

Fixed in build148

Note: See TracTickets for help on using tickets.

Download in other formats: