gpsdrive's Mapnik doesn't start since stuff changed

[hamish]

Hi,

gpsdrive runs Mapnik to generate map tiles on-the-fly using the OSM data loaded into PostGIS and shapefiles from Natural Earth for the coastline, world city names, etc.

This failed in beta1.iso (cleanly, it just reverts to static satellite image backdrop) since the shapefiles moved, specifically 10m_land.shp isn't where it used to be and had a name change. I've fixed those in SVN now, but the transition to PostGIS 2 now causes a crash:

terminate called after throwing an instance of 'mapnik::datasource_exception'
  what():  :
ERROR:  function asbinary(geometry) does not exist
LINE 1: SELECT AsBinary("way") AS geom,"aeroway","amenity","building...
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.
Full sql was: 'SELECT AsBinary("way") AS geom,"aeroway","amenity","building","highway",\
"landuse","leisure","man_made","military","natural","power","railway","tourism" from \
 (select * from planet_osm_polygon order by z_order,way_area desc)\
 as leisure WHERE "way" && SetSRID('BOX3D(-134041.6 6972313.6,-130457.6 6975180.799999999)'::box3d, 900913)'

Aborted (core dumped)

hopefully a PostGIS expert can provide some tips, I'm not seeing any asbinary() or AsBinary() in the gpsdrive source code.

​http://gpsdrive.svn.sourceforge.net/viewvc/gpsdrive/trunk/src/database_postgis.c?revision=HEAD&view=markup

​http://gpsdrive.svn.sourceforge.net/viewvc/gpsdrive/trunk/src/database_postgis.h?revision=HEAD&view=markup

in the mean time I've just set mapnik=0 in the ~/.gpsdrive rc file (see gisvm/bin/install_gpsdrive.sh) to disable mapnik mode.

thanks for any pointers, Hamish

[hamish]

this one's probably more relevant:

​http://gpsdrive.svn.sourceforge.net/viewvc/gpsdrive/trunk/src/database.c?revision=2703&view=markup

[hamish]

Sergio noted on the ML that "legacy.sql" could be used to work around the problem. If you know about that, please post more to this ticket.

​http://lists.osgeo.org/pipermail/live-demo/2013-January/006875.html

thanks, Hamish

[hamish]

fwiw "There is an alternative legacy_minimal.sql you can run instead which will install barebones needed to recover tables and work with apps like MapServer and GeoServer. If you have views that use things like distance / length etc, you'll need the full blown legacy.sql"

​http://www.postgis.org/docs/postgis_installation.html

[darkblueb]

SetSRID() gets changed to ST_SetSRID() on line #454 of ​http://gpsdrive.svn.sourceforge.net/viewvc/gpsdrive/trunk/src/database.c

but, since the call that surrounds it is ST_DWithin(), it looks like someone was already doing things the standards 1.5 way, they just missed converting the SetSRID() call too...

it appears that the rest of that file calls only db_sqlite_query(), but db_streets_get() is special (?)

[hamish]

perhaps it is this:

​https://github.com/mapnik/mapnik/issues/1083 ​http://trac.osgeo.org/postgis/ticket/833

note there are a number of GeomFromText in

gpsdrive/scripts/osm/perl_lib/Geo/OSM/SegmentList.pm

which may still be a problem? (if so can 'sed -i -e' them)

legacy_minimal.sql may still be needed however.

Hamish

[hamish]

btw, new iteration of the gpsdrive package now uploaded with SetSRID changed to ST_SetSRID in the C code.

[hamish]

Hi, after some more digging around on the beta1 iso and brainstorming with Brian in IRC, we seem to have tracked it down.

On the disc there is both a libmapnik2-2.0 from Ubuntu's official 12.04 release, and a libmapnik from mapnik's ppa.launchpad.net repo.

The version from the ppa is newer, the official 2-2.0 version didn't have all the postgis 2.0 kinks out of it yet at time of release and includes the AsBinary() call.

compare ubuntu's libmapnik2-2.0's

beta1iso$ strings /usr/lib/mapnik/2.0/input/postgis.input | grep Binary

with mapnik ppa's libmapnik's

beta1iso$ strings /usr/lib/mapnik/input/postgis.input | grep Binary

legacy_minimal.sql does contain the AsBinary() call, so will probably help. Another idea is to rebuild the gpsdrive .deb package based on the mapnik ppa libmapnik-dev, but if possible I'd hope to avoid the 3rd party dependency, it leads to dll hell.

Hamish

[hamish]

Hi,

some observations and updates:

legacy_minimal.sql is now called for the osm psql dbs, which allows GpsDrive's maps to work again. That change has been applied in svn.

Kosmo needs to have the full legacy.sql loaded into the natural_earth2 psql db in order for it to display map features (table load still seems to work there). we thought is was a call to extent(), but that shows up in the /var/log/postgress/ error log even after applying PostGIS's legacy.sql, so would seem to be unrelated. I'd note that the visibility tick box is still hard to manipulate in Kosmo; see quickstart for a workaround. The legacy.sql has *not* been applied to natural_earth2 db in svn yet. The good news is that if it causes any problems (and I'm not sure if it would or not) you can apply it at run time, so worst case perhaps even as part of the Kosmo.desktop startup icon's Exec= command?

many of the quickstarts were not updated for the new natural earth db name and shapefile name variationss (.please. don't change those unnecessarily...). I fixed all the English ones I could find, but the translations will need that too. Same with mapnik -> mapnik2 python library in the Mapnik quickstart.

regards, Hamish

[darkblueb]

I do not understand how or why Kosmo can shift the base system for all other apps - I read above that GPSDrive is now (thankfully) resolved.

[hamish]

Replying to darkblueb:

I do not understand how or why Kosmo can shift the base system for all other apps -

I don't think it's as bad as all that, legacy.sql seems to be a bolt-on plugin to help with apps in transition, not a replacement of anything.

I read above that GPSDrive is now (thankfully) resolved.

"worked around" is probably the better way of putting it.

the current solution for Kosmo is that the legacy.sql gets applied as part of the Kosmo startup script. There's no check for if that's been done before (a little help please?) so if Kosmo is run multiple times it will get applied multiple times. (maybe that's harmless?)

there's a very good chance something else on the disc will also not be PostGIS 2.0 ready; we'll just have to wait and see.

regards, Hamish

[hamish]

I wrote:

the current solution for Kosmo is that the legacy.sql gets applied as part of the Kosmo startup script.

i.e. at run time as part of /usr/local/bin/kosmo.sh, not at build time as part of install_kosmo.sh.

[sbcalvo]

Replying to darkblueb:

I do not understand how or why Kosmo can shift the base system for all other apps - I read above that GPSDrive is now (thankfully) resolved.

It's not only a "Kosmo problem" that shift the base system for all the other apps, I did notice it by checking the Kosmo Desktop quickstart and warned that the change to PostGIS 2.0 could affect to other applications as well.

As the matter of fact, it does, not only Kosmo Desktop is not able to load the PostGIS tables, but also gvSIG and gpsdrive have a similar problem. I did some tests and gvSIG complains about the asewk(geometry,unknow) function when trying to load a table from natural_earth2 database. Maybe they are the unique to fail, but I'm not sure about it.

The legacy.sql is a solution for the problem that would solve for everyone for sure, as it restores all the removed functions. Kosmo Desktop needs only the asewkb, extent and geometryfromtext functions to be present at the database, legacy_minimal.sql is not enough.

Of course, I could change Kosmo Desktop to use the new functions (as it will do in the next release), but it won't solve the problem for the others.

[kalxas]

fixed and confirmed in [9640]

[hamish]

So gvSIG needs it too? :-/

I committed a hack yesterday to get Kosmo to work (r9639) by loading legacy.sql in its startup script. If it's gvSIG (and perhaps other stuff too), the best I can offer is this not entirely popular patch:

Index: bin/load_gisdata.sh
===================================================================
--- bin/load_gisdata.sh	(revision 9640)
+++ bin/load_gisdata.sh	(working copy)
@@ -153,7 +153,7 @@
 sudo -u $POSTGRES_USER createdb natural_earth2
 sudo -u $POSTGRES_USER psql natural_earth2 -c 'create extension postgis;'
 sudo -u $POSTGRES_USER psql natural_earth2 \
-  -f /usr/share/postgresql/9.1/contrib/postgis-2.0/legacy_minimal.sql
+  -f /usr/share/postgresql/9.1/contrib/postgis-2.0/legacy.sql
 
 for n in "$SRC_DIR"/*.shp;
 do

[hamish]

Replying to hamish:

the best I can offer is this not entirely popular patch:

applied with r9650.