include postgis 1.4

[hamish]

We all agree it would be great to include PostGIS 1.4 on the live DVD. The question is when.

I now realize that there isn't a ticket for this (the one I had in the back of my mind was the wish for Postgres 8.4). As there is ongoing unhappiness about this situation (and we want our valued colleagues to be happy; this is supposed to be fun), and the discussion on-list has stalled, I'm creating a ticket for it. For anyone following the list there is very little new here; just summarizing.

There is already PostGIS 1.3 installed on the disc and it is used by a couple of projects. 1.4 would be a concurrent upgrade to that on its own for folks to explore. There is nothing inherently broken about 1.3 besides that it does not have all of the latest exciting features.

1.4 is only yet packaged for Debian in Experimental for two weeks (since long time maintainer Fabio orphaned it); it is not yet into Debian/Unstable or Testing, and not yet into Ubuntu anything (which IIUC filters in from Testing). Read all about it:

​http://packages.debian.org/experimental/postgis

Note this is nothing to do with the code quality or readiness of the PostGIS source code; this has everything to do with how the compiling and packaging interacts with the rest of the system and other packages, and the livedvd freeze and release cycle.

As posted to the mailing list I'd like to see a few things before I'd be satisfied that it would be safe to be included on the next round of new features. These are:

• build the complete live dvd from the latest SVN and post the build logs for review. (to show that it does not break any other package at build-time.)
  • this build log can now be found in beta5; looks clean enough. next step is user run-time testing.

• get the blessing of the pgRouting and Mapnik folks. (ie the other PostGIS stakeholders)
  • some responses in this thread: ​http://thread.gmane.org/gmane.comp.gis.osgeo.livedemo/613/

• a listing of what 'make install' copies into the filesystem. e.g. as I understand it there will be two different versions of shp2pgsql and pgsql2shp installed, v1.3 in /usr/bin, v1.4 in /usr/local/bin; and to show that all .so-names are unique.
  • Still waiting to see a patch and install mainifest. Known are bin/shp2pgsql and bin/pgsql2shp and their man pages. Possible conflict with shared lib of the same name from 1.3 if liblwgeom.a gets installed (I don't think it is, but not sure). A patch should be submitted renaming those programs to eg shp2pgsql14, and it must be shown that they statically link against it, and that other packages' software will not try to statically link against liblwgeom.a. Other conflicts?

In general I would follow DebianGIS's packaging as a guide, and much prefer to use their Postgres 8.4 and PostGIS 1.4 packages directly once they hit the main archives, and rather move this discussion over there. see

​http://svn.debian.org/viewsvn/pkg-grass/packages/postgis/trunk/debian/

The wheels are already in spin for it to be in the next cut, just not this one. No project is immune from teething problems.

regards, Hamish

[hamish]

Replying to hamish:

As posted to the mailing list I'd like to see a few things before I'd be satisfied that it would be safe to be included on the next round of new features. These are:

before I get yelled at, the "next round of features" comment is not as stated before, but due to the alphas & betas it is now my current thinking.

Hamish

[wildintellect]

I was just pondering, if postgis 1.4 gets turned on, what if some application wants a db with 1.3. Should we be creating 2 template databases for the time being?

Maybe this is moot if other apps only rely on the 1.3 libs being somewhere, and all the SQL is 100% backwards compatible.

[hamish]

ldd will tell you which shared libraries are used and where the program is looking for them. for shared libs the two PostGIS so-names are distinct:

/usr/lib/postgis/1.3.3/postgres/8.3/lib/liblwgeom.so.1.3 and .../libpostgis-1.4.so.0.0

but the 1.4 static lib name matches 1.3's soname. unsure what that could lead to if compiles try to use static libs; I don't think the .a is installed by 1.4, but not sure about that.

[the next bit is slightly theoretical as it's impossible for me to try all build permutations of the varying software, and every machine will be slightly different. All I know is that I've fought with enough multiple installs of e.g. GDAL over the years to have become highly fastidious and very wary about allowing multiple versions of the same library on the same production machine]

For any of the apps that compile after PostGIS 1.4 is installed, which version of PostGIS is used may depend on which one ./configure|cmake finds first in the LD library path, in a similar way to the issues that follows. I can't answer this for sure, autoconf is way too complicated for me to audit properly. it's a run it and pray situation.

The obvious breakage using the proposed change as-is will be if the user follows one of a hundred tutorials out there on the web:

createdb -O username gisdb
psql -d gisdb -f /usr/share/postgresql-8.3-postgis/lwpostgis.sql
...
shp2pgsql -a roads gisdb ...
# or whatever

but because /usr/local/bin often comes before /usr/bin in the search $PATH, the 1.4 ver of shp2pgsql will be used and then it tries to insert 1.4isms into a 1.3 template ... and the most likely result is kerplunk.

(again, not tested, but it's a compelling theory)

so the known needed patch is to remove/rename/move away/whatever those 2 binaries and their man pages, and without an install manifest I can't say what else, if anything. Due to the way that this has been pushed in svn at such a late stage of the release cycle, without (or in disregard of) list consensus, and with the only justification given being that the new version is faster and has better docs, my motivation to work on such things is now gone. I've diverted enough energy away from positive pursuits already.

IMHO this is a lot of headaches and auditing work which instantly goes away the minute the now 2 week old official debian package hits the main archives.

said my piece and walking away, Hamish

[darkblueb]

well, I've been the primary proponent of a 1.4 include, as people on the IRC know by now.. I wrote the stick figure script that pulled in a source tarball, compiled and installed it. It was very professionally enhanced by Hamish soon thereafter. I don't agree with the characterization that it was thrown in "late in the process" as when I wrote that, many full LiveDVD packages were completely dangling and dysfunctional still.

I am a little overwhelmed by the attention to detail.. its a plug-in, not a lib. So it behaves as a "leaf" element, and has far less compatability implications than a lib, as far as I can see. However, given the a high quality product is everyone's goal, thorough is almost invariably better.

I think there is a misconception running through the minds of many about how PostGIS is actually invoked. True it is a fundamental piece of technology. In no small part why I felt it was so important to have on the FOSS4G 2009 disk. But lets take two examples: OGR and the template.

As far as I know, the OGR Pg driver is a libpq client of Postgres. That is, a connection is made via libpq and a DSN string. Nothing touches, links to, refers to, stands on top of, warmly shakes hands with or buys beer for any PostGIS anything, ever.

Each database is its own world, by design. Think of a huge, high security shared installation. All Pg functions, including those that call actual .so code, are woven into the individual database through the Pg load process. PostGIS is just a collection of SQL and functions, most of which call a particular .so library on *nix, isolated in a database.

As far as using the Postgis template, and needing one for 1.3 and 1.4, I think, with all due respect, thats simply splitting hairs. There is no standard for a PostGIS tempate.. any template that exists is up to whatever creates it and whatever else names it. That include PostGIS version used.. As far as I know, 1.3 users are 99.98% compatible with 1.4 in practice. From anecdotal observation, I think any 1.3 user would be happy to move to 1.4. Upgrades are hard enough in the stodgy, risk-averse database world. Lets not exaggerate the picture if at all possible..

Lastly, as far as the shp2pgsql binary.. This is the one legitimate issue as far as I am concerned.. and as already pointed out, I raised it within 1 hour of writing the original script. My personal time and skill level was not up to the kind of analysis shown here by others as things unfolded. It was simply not tested to the satisfaction of one particular party, who unilaterally put a hold in the inclusion of PostGIS 1.4 on the disk. which yes, bugs me.. But they say, choose your battles... All in all this is a fantastic product, this LiveDVD, and a real thanks goes out to all, myself and my loyal adversary and everyone, for coming together to make this happen. It was tons of work to get a final. Everyone gets a little frazzled, myself included. I strive to maintain a civil approach, even as technical style and contributions sometimes do not mesh perfectly.

I also look forward to PostGIS 1.4, especially the manual, to be prominently part of this excellent OSGeo production. I predict this LiveDVD will be very, very popular and influential in the near future. $0.02

[darkblueb]

of course OGR uses PostGIS functions, but it does not link or call the lib directly at all, is what I meant above..

[hamish]

ok, UbuntuGIS has added it to their repo for Karmic, and Alex has adjusted the install script in trunk to match. PostGIS 1.4 is already in Lucid, so in future this will get even easier.

still todo:

• testing! probably some old postgis 1.3 commands still lurk.
• update medford sample dataset and load_postgis.sh

Hamish