chroot build method: mapfish does not start

[kalxas]

There is a 500 error when attempting to start Mapfish in beta9 (live-session)

[hamish]

in the chroot method, when the user account was deleted the /etc/group file was purged as well (or some similar timeline of events).

the result was that 'user' was not a member of www-data, tomcat6, users, staff, etc anymore. potential work-around committed in r8913. maybe it fixes it, maybe it don't.

Hamish

[kalxas]

still present after the fix in build8915

[hamish]

rc.local fix didn't take. re-fixed in r8928.

even with the groups updated I still get an Internal Server Error when I go to the mapfish example listed in the quickstart.

same "FATAL: database "v2.2_mapfishsample" does not exist" error as reported earlier.

/var/log/apache2/error.log says:

Fri Aug 10 08:52:19 2012] [notice] Graceful restart requested, doing restart
apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
[Fri Aug 10 08:52:22 2012] [warn] mod_wsgi: Compiled for Python/2.7.2+.
[Fri Aug 10 08:52:22 2012] [warn] mod_wsgi: Runtime using Python/2.7.3.
[Fri Aug 10 08:52:22 2012] [notice] Apache/2.2.22 (Ubuntu) mod_fcgid/2.3.6 PHP/5.3.10-1ubuntu3.2 with Suhosin-Patch mod_wsgi/3.3 Python/2.7.3 configured -- resuming normal operations
[Fri Aug 10 08:52:25 2012] [notice] Graceful restart requested, doing restart
apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
[Fri Aug 10 08:52:26 2012] [warn] mod_wsgi: Compiled for Python/2.7.2+.
[Fri Aug 10 08:52:26 2012] [warn] mod_wsgi: Runtime using Python/2.7.3.
[Fri Aug 10 08:52:26 2012] [notice] Apache/2.2.22 (Ubuntu) mod_fcgid/2.3.6 PHP/5.3.10-1ubuntu3.2 with Suhosin-Patch mod_wsgi/3.3 Python/2.7.3 configured -- resuming normal operations
[Fri Aug 10 20:05:59 2012] [warn] mod_wsgi: Compiled for Python/2.7.2+.
[Fri Aug 10 20:05:59 2012] [warn] mod_wsgi: Runtime using Python/2.7.3.
[Fri Aug 10 20:05:59 2012] [notice] Apache/2.2.22 (Ubuntu) mod_fcgid/2.3.6 PHP/5.3.10-1ubuntu3.2 with Suhosin-Patch mod_wsgi/3.3 Python/2.7.3 configured -- resuming normal operations
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1] mod_wsgi (pid=3265): Target WSGI script '/usr/local/lib/mapfish/MapfishSample/buildout/parts/modwsgi/wsgi' cannot be loaded as Python module.
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1] mod_wsgi (pid=3265): Exception occurred processing WSGI script '/usr/local/lib/mapfish/MapfishSample/buildout/parts/modwsgi/wsgi'.
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1] Traceback (most recent call last):
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/parts/modwsgi/wsgi", line 46, in <module>
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     application = loadapp("config:" + configfile)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/PasteDeploy-1.3.4-py2.7.egg/paste/deploy/loadwsgi.py", line 203, in loadapp
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     return loadobj(APP, uri, name=name, **kw)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/PasteDeploy-1.3.4-py2.7.egg/paste/deploy/loadwsgi.py", line 224, in loadobj
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     return context.create()
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/PasteDeploy-1.3.4-py2.7.egg/paste/deploy/loadwsgi.py", line 617, in create
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     return self.object_type.invoke(self)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/PasteDeploy-1.3.4-py2.7.egg/paste/deploy/loadwsgi.py", line 109, in invoke
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     return fix_call(context.object, context.global_conf, **context.local_conf)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/PasteDeploy-1.3.4-py2.7.egg/paste/deploy/util/fixtypeerror.py", line 57, in fix_call
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     val = callable(*args, **kw)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/mapfishsample/config/middleware.py", line 37, in make_app
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     config = load_environment(global_conf, app_conf)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/mapfishsample/config/environment.py", line 49, in load_environment
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     init_model(engine)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/mapfishsample/model/__init__.py", line 16, in init_model
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     class Poi(Base, GeometryTableMixIn):
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/ext/declarative.py", line 1165, in __init__
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     _as_declarative(cls, classname, cls.__dict__)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/ext/declarative.py", line 1064, in _as_declarative
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     **table_kw)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/schema.py", line 265, in __new__
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     table._init(name, metadata, *args, **kw)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/schema.py", line 324, in _init
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     self, include_columns
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/engine/base.py", line 2262, in run_callable
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     conn = self.contextual_connect()
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/engine/base.py", line 2328, in contextual_connect
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     self.pool.connect(),
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/pool.py", line 209, in connect
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     return _ConnectionFairy(self).checkout()
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/pool.py", line 370, in __init__
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     rec = self._connection_record = pool._do_get()
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/pool.py", line 696, in _do_get
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     con = self._create_connection()
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/pool.py", line 174, in _create_connection
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     return _ConnectionRecord(self)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/pool.py", line 255, in __init__
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     self.connection = self.__connect()
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/pool.py", line 315, in __connect
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     connection = self.__pool._creator()
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/engine/strategies.py", line 80, in connect
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     return dialect.connect(*cargs, **cparams)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]   File "/usr/local/lib/mapfish/MapfishSample/buildout/eggs/SQLAlchemy-0.7.2-py2.7-linux-i686.egg/sqlalchemy/engine/default.py", line 275, in connect
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]     return self.dbapi.connect(*cargs, **cparams)
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1] OperationalError: (OperationalError) FATAL:  database "v2.2_mapfishsample" does not exist
[Fri Aug 10 20:09:44 2012] [error] [client 127.0.0.1]  None None

Hamish

[hamish]

Hi,

Brian and myself have been working on this one a bit, and we have a theory about what's happening.

exec summary: the data base was not created correctly; it's not mapfish's fault. inchroot.sh is setting LC_ALL=C which is propagating to install_postgis.sh and the postgres templates are being created as SQL_ASCII encoding instead of UTF8 encoding. Mapfish was expecting it to be UTF8.

solutions include: not having inchroot do that, having install_postgis.sh unset/reset the LANG variables at the beginning of its script, and/or explicitly stating the encoding type at each createdb step on the liveDVD.

We'll wait to hear why inchroot was doing that before touching it, have added some debug messages to install_postgis.sh to get a clearer idea of what's happening there, and Brian's gone around and added "-E UTF8" in a number of places where it wasn't explicitly stated.

---

from yesterday's 6.0-build-r8942 build log:

...
*************** PICKED VERSIONS ****************
[versions]

*************** /PICKED VERSIONS ***************
createdb: database creation failed: ERROR:  new encoding (UTF8) is incompatible with the encoding of the template database (SQL_ASCII)
HINT:  Use the same encoding as in the template database, or use template0 as template.
...

---

in the 5.5 release all postgres tables were encoded as UTF8, same for 6.0beta7 (the last of the remastersys line); we checked.

For yesterday's 6.0-build-r8942 however:

user@osgeolive:~$ psql -l
                                 List of databases
       Name       |  Owner   | Encoding  | Collate | Ctype |   Access privileges   
------------------+----------+-----------+---------+-------+-----------------------
 52nSOS           | postgres | UTF8      | C       | C     | =Tc/postgres         +
                  |          |           |         |       | postgres=CTc/postgres+
                  |          |           |         |       | user=CTc/postgres
 RASBASE          | user     | SQL_ASCII | C       | C     | 
 mapbender        | user     | UTF8      | C       | C     | 
 natural_earth    | user     | UTF8      | C       | C     | 
 osm_local        | user     | UTF8      | C       | C     | 
 osm_local_smerc  | user     | UTF8      | C       | C     | 
 petascopedb      | user     | SQL_ASCII | C       | C     | 
 pgrouting        | user     | SQL_ASCII | C       | C     | 
 postgres         | postgres | SQL_ASCII | C       | C     | 
 sahana           | sahana   | SQL_ASCII | C       | C     | 
 template0        | postgres | SQL_ASCII | C       | C     | =c/postgres          +
                  |          |           |         |       | postgres=CTc/postgres
 template1        | postgres | SQL_ASCII | C       | C     | =c/postgres          +
                  |          |           |         |       | postgres=CTc/postgres
 template_postgis | user     | UTF8      | C       | C     | 
 user             | user     | SQL_ASCII | C       | C     | 
(14 rows)

cheers, Hamish

[hamish]

hopefully fixed by r8996. (unset LC_ALL in install_postgis.sh)

Hamish

[kalxas]

confirmed in beta10