git builds always need upgrade

[komzpa]

We're building 2.4.0 from git.

postgis_scripts_installed() lists it as '2.4.0dev'; postgis_scripts_released() lists it as '2.4.0dev r0'.

Standard upgrade dev → devnext → dev passes, but still needs upgrade.

[robe]

This might not be a global issue, might be something specific with how komzpa is setup. Discussed on IRC with him.

Travis who also builds from github does show revision numbers. Though I forget what magic strk put in place to make that happen.

PostgreSQL 9.6.5 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4, 64-bit

  Postgis 2.4.0dev - r15801 - 2017-09-23 13:35:03

  scripts 2.4.0dev r15801

  raster scripts 2.4.0dev r15801

  GEOS: 3.5.0-CAPI-1.9.0 r4084

  PROJ: Rel. 4.8.0, 6 March 2012

  GDAL: GDAL 2.1.0, released 2016/04/25

winnie also build from git (git.osgeo.org) and she shows revision numbers as well.

I'm going to install the extension files from winnie to make sure it's not happening during the generation of the extension files.

Komzpaa, are you absolutely sure your make install is working and installing latest scripts. As I mentioned on IRC, even if revision number wasn't coming thru, I should still at least see a PGSQL=100 since PostGIS 2.4.0beta. Yours is not showing that which makes me question if they are the latest.

— dialog from IRC below

07:38:23	Komzzpa:	POSTGIS="2.4.0dev" GEOS="3.7.0dev-CAPI-1.11.0 0" SFCGAL="1.3.1" PROJ="Rel. 4.9.3, 15 August 2016" GDAL="GDAL 2.1.2, released 2016/10/24" LIBXML="2.9.4" LIBJSON="0.12.1" (core procs from "2.4.0dev" need upgrade) RASTER (raster procs from "2.4.0dev" need upgrade) (sfcgal procs from "2.4.0dev" need upgrade)
07:38:54	Komzzpa:	alter extension postgis update to '2.4.0devnext'; alter extension postgis update to '2.4.0dev'; does not help to get rid of "need upgrade"
07:39:06	Komzzpa:	what could we have broken?
08:03:33	robe2:	Komzzpa hmm that is odd where were you upgrading from I thought it lists the r version though perhaps coming from git it doesn't
08:04:35	robe2:	Komzzpa mine lists the svn version - POSTGIS="2.4.0dev r15800" PGSQL="100" GEOS="3.7.0dev-CAPI-1.11.0 316c4bc" SFCGAL="1.3.1" PROJ="Rel. 4.9.1, 04 March 2015" GDAL="GDAL 2.2.1, released 2017/06/23" LIBXML="2.7.8" LIBJSON="0.12" LIBPROTOBUF="1.2.1" RASTER
08:04:57	robe2:	it's very strange yours doesn't list PGSQL
08:05:54	robe2:	you sure latest scripts got installed? perhaps the binaries got installed but the scripts didn't that would trigger that issue
08:06:33	Komzzpa:	we've found that git builds make postgis_scripts_released='2.4.0dev r0' and postgis_scripts_installed='2.4.0'
08:07:27	Komzzpa:	should I file a ticket?
08:07:46	robe2:	Komzzpa hmm that's annoying. Yah that would be good
08:08:06	robe2:	I think we might be looking at revision numbers though probably moving forward we'd have to stop that.
08:08:54	robe2:	still even given that you should be seeing PGSQL=... in your output if you are installing latest scripts
08:09:44	robe2:	A bug I found in earlier is the PGSQL compare was wrong so if you PGSQL you are using wasn't compiled with same gcc etc. as what you are runnig, it gave a bogus need upgrade message
08:10:11	robe2:	at very least it should list PGSQL ... which means you don't have latest scripts installed.
08:10:17	Komzzpa:	I think it's one of beta's
08:10:43	Komzzpa:	I bumped into it exactly when tried to figure out which one :)
08:11:11		* robe2 is going to miss revision numbers when we move to git
08:11:17	robe2:	increasing that is
08:11:39	Komzzpa:	you can implement commit counting :)
08:11:56	robe2:	Komzzpa oh how do you do that
08:12:42	Komzzpa:	kom@junocat postgis % git log | grep commit | wc -l
08:12:43	Komzzpa:	11803
08:12:49	Komzzpa:	in a brainless way
08:13:11	Komzzpa:	probably some sophisticated exists
08:14:12	Komzzpa:	it won't be precisely comparable in branches
08:17:16	robe2:	Komzzpa though I have winnie building from git.osgeo.org and she has r versions - http://winnie.postgis.net:1500/job/PostGIS_EDB_Regress_winnie/5617/consoleFull
08:17:17	sigq:	Title: PostGIS_EDB_Regress_winnie #5617 Console [Jenkins] (at winnie.postgis.net:1500)
08:17:45	robe2:	not sure how that's happening unless building from github is different from git.osgeo.org (and strk has something cooked up there)
08:18:47	Komzzpa:	#POSTGIS_SVN_REVISION=passed_in_by_buildbot
08:18:58	Komzzpa:	this may be a builder's trick
08:21:32	robe2:	Komzzpa hmm I think I had put that in a while ago though I don't see it being used and where I'm passing it in anymore
08:23:08	robe2:	Komzzpa travis has revision number too and he builds from git - https://travis-ci.org/postgis/postgis?branch=svn-trunk
08:23:09	sigq:	Title: Travis CI - Test and Deploy Your Code with Confidence (at travis-ci.org)
08:24:58	Komzzpa:	hmm
08:25:19	Komzzpa:	may it be that it is chopped off somehow on deb build...

[robe]

okay not seeing an issue with extension building.

I used winnie's postgis 2.4.0alpha build and SELECT postgis_full_version(); showed alpha and the r number.

Then I copied over the latest dev build and did

ALTER EXTENSION postgis UPDATE;

POSTGIS="2.4.0dev r15800" PGSQL="100" GEOS="3.7.0dev-CAPI-1.11.0 81c6d4a" PROJ="Rel. 4.9.1, 04 March 2015" GDAL="GDAL 2.2.1, released 2017/06/23" LIBXML="2.7.8" LIBJSON="0.12" LIBPROTOBUF="1.2.1" RASTER

Her PostGIS 2.4.0 source for builds are pulled from git.osgeo.org/gogs

I looked at the tarball for PostGIS 2.4.0rc1 that debbie built which I used git.osgeo.org/gogs to build from and that has the revision number in it as well.

Komzzpa — what does the file postgis_svn_revision.h read? I suspect in your case it reads 0

which means the routine that pulls the revision number utils/svn_repo_revision.pl is not working for you.

There is logic to read both svn and git in that script. Looks like for git it uses git log

${git_exe}\" log --grep=git-svn -1 | grep git-svn | cut -d@ -f2 | cut -d' ' -f1

Note that with each mirror this gets tacked on to the git log

— this is the last one

git-svn-id: http://svn.osgeo.org/postgis/trunk@15802 b70326c6-7e19-0410-871a-916f4a2858ee

So that command to parse out the number after @ probably isn't working for you.

[robe]

Just occurred to me what might be Komzzpa problem. Since we pull the svn revision number from the git-svn-id, any git branch that is not in svn would not have a revision number and then I think it would exhibit the issue he is seeing.

Since Debbie and Winnie only build by the mirrored branches, I wouldn't be seeing the issue in them.

[robe]

okay that's not it. I built from git in a checked out experimental branch and still got revision numbers. Maybe git log just doesn't work on Komzzpa's system.

I should also add I looked at pull requests run from travis of Komzzpa's and they have revision numbers too like this pull request regress run - ​https://travis-ci.org/postgis/postgis/builds/286578260?utm_source=github_status&utm_medium=notification

[gsmol]

If you try to build without .git directory you`ll get 'r0' revision number. Is it possible to store revision number as macros?

[robe]

I suspect the same could be said about building from svn. That's what we have tarballs for since those already have the revision number stamped.

​https://postgis.net/source/

I don't see how we could accomplish without having our build bot stamping all the time.

Eventually once it's in git, we'd probably just switch to using git hashes anyway, though would that work without having .git folder? Not having a .git to me would defeat the purpose of building from git in the first place.

[strk]

Confirmed: either you build from a tarball (which should have the revision already provided) or you build from a GIT (with .git dir) or SVN (with .svn dir) checkout. Other kind of builds are not expected to ever yeld anything but "0" as the revision number. Besides, if "0" is consistent between library and scripts you should still not get the "needs upgrade" message, so that message is still meaningful as it's telling you that scripts and libs are coming from different builds.

[strk]

re-reading the original ticket description I see there's a lack of "r" in one of the outputs. So this is a real bug. It's due to this snippet in sqldefines.h:

#if POSTGIS_SVN_REVISION
#define _POSTGIS_SQL_SELECT_POSTGIS_SCRIPTS_VERSION $$ SELECT '2.5.0dev'::text || ' r' || POSTGIS_SVN_REVISION::text AS version $$
#else
#define _POSTGIS_SQL_SELECT_POSTGIS_SCRIPTS_VERSION $$ SELECT '2.5.0dev'::text AS version $$
#endif

That logic (considering r0 as special) is not present in the implementation of postgis_scripts_installed, thus the discrepancy and the misleading warning.

[strk]

In 16117:

Always include revision when defined (even if 0)

Should fix #3863 (scripts always need upgrade)

[strk]

Komzpa please see if r16117 makes your environment happier