Changes between Initial Version and Version 1 of PostGISExtensionPaths

Timestamp:

Feb 11, 2022, 10:15:29 AM (2 years ago)

Author:

robe

Comment:

—

PostGISExtensionPaths

@@ +1 @@
+= RFC-6: Managing PostGIS Extension Paths with Fewer Files =
+||Date:|| 2022/02/11||
+||Author:|| Regina Obe||
+||Last Edited:|| 2022/02/11||
+||Last Updated By: || Regina Obe <lr at pcorp dot us> ||
+||Status:|| In Discussion||
+
+== PROBLEM STATEMENT ==
+With each new version of PostGIS, we need to ship more and more extension upgrade scripts.
+In reality all the upgrade scripts are just a symlink to the main one or exact copies of the main one.
+This results in a users extensions folder cluttered with 100s of PostGIS related scripts.
+
+Not only that, but for each new version we ship, we always need to remember to add the recently shipped micros of other versions
+to the extensions/updateable.mk script.
+
+The largeness of the script files (when not symlinked, which is often the case), is the biggest issue of all, and will become even more of an issue
+with shipping spatial_ref_sys updates.
+
+The solution of the above issues is to have: Solution 1, Solution 2, and Solution 3 implemented, as some solutions are not backward compatible but do provide extra benefits.
+
+
+== SOLTUION 1 ==
+
+This has already been committed in master branch for PostGIS 3.3.0.
+Embedded in:
+
+
+----
+SELECT postgis_extensions_upgrade();
+----
+
+What it does:
+Update the current version to ANY and then run the updates.
+This has the affect of always forcing the  postgis--ANY--<current version>.sql script to run which handles all versions.
+
+Benefits of this approach:
+
+1. We only need to ship one extension upgrade script file per extension.  The postgis--ANY--<current version>.sql
+2. No clutter of 100s of 1000s of files.
+3. We don't need to maintain extensions/updateable.mk and remember to update it.
+4. It is possible to upgrade from postgis (older PostgreSQL Version) 3.3.0 to (new PostgreSQL version)  3.3.0. This is generally what is needed in pg_upgrade.
+
+**Drawbacks:** 
+
+1. People can only upgrade using 
+
+```
+SELECT postgis_extensions_upgrade();
+```
+Or do the manually mock with the pg_catalog.pg_extension table themselves
+
+2. We are mucking with system tables, pg_catalog.pg_extension, which we have done for other things, but is a questionable practice.
+
+3.  Packagers who have put in nice features to auto upgrade extensions during pg_upgrade, now have to treat us a special snowflakes.
+
+
+== SOLUTION 2 ==
+
+Beg and plea upstream to recognize a % or ANY as a first class citizen to mean any prior version.
+
+This approach allows for defining a new line item in the control file, that denotes something like
+
+
+----
+upgrade_script_pattern = %--<version>
+----
+
+Where the % will be treated as a wildcard to match to provider the best match.
+
+**Benefits of this approach:**
+
+1. We have made what we are doing, built into the postgis machinery, so no special care needs to be done by packagers or users.
+It makes us not a special snowflake.
+
+2.  It will help other extensions which have similar issues and just want a single upgrade script. We expect many extensions to benefit, though we will benefit the most.
+
+**Drawbacks:**
+
+1. Old PostgreSQL versions, won't understand this new world order, so we still need to maintain a backward compatible way for 5 years or more.
+
+
+=== SOLUTION 3 ===
+
+The 0-byte file and strip model.  This approach creates 0 or very small byte files, with their existing solely to maintain PostgreSQL extension chain model.
+To reduce file bloat, we will only support the last two micros of any release.  Anyone who isn't in the latest micro (or latest to last micro) of each minor/major, with need to upgrade to said version OR use the postgis_extensions_upgrade() function.
+
+In this model, our extension files look like below:
+An additional addendum to this SOLUTION 3, is to not maintain - extensions/updateable.mk but instead have this file generated as part of the build process and be shipped as part of the tar ball. 
+
+The extensions/updateable.mk will be generated by looking at git tags all the prior versions, and picking the last two of each mino.
+
+our generated extension upgrade scripts will look like the below, with only the postgis--3.3.0MAX--3.3.0.sql having any upgrade statements in it.
+
+----
+:
+postgis--3.1.4--3.3.0MAX.sql
+postgis--3.1.5--3.3.0MAX.sql
+postgis--3.2.0--3.3.0MAX.sql
+postgis--3.2.1--3.3.0MAX.sql
+postgis--3.3.0MAX--3.3.0.sql
+----
+**Benefits of this approach:**
+
+1. We are staying between the lines. Respecting known practices, but still having much lighter weight scripts.
+2. A little less clutter if we include the addendum (of only keeping the last 2)
+3. A little little easier release cycle if we don't need to maintain extensions/updateable.mk and remember to update it.
+4. No change in upgrade extension workflow for packagers (or users who think of PostGIS as any other extension).
+5. It will work for all PostgreSQL versions supported
+
+** Downsides:**
+
+1. We still have some clutter of files, and without addendum, we have just as much clutter of files as before.
+2. With addendum, it is not possible for a user to upgrade using ALTER EXTENSION from a version we don't ship a script for. They would have to use and that assumes this version -- has the ANY/MAX hack.
+
+----
+SELECT postgis_extensions_upgrade();
+----
+
+3. It is not possible to upgrade from version (prior PostgreSQL version) 3.3.0 to (newer PostgreSQL version) 3.3.0 which is needed often in pg_upgrade to gain the benefits of new features in latest PostgreSQL version that are not exposed in older PostgreSQL versions.  Such operation would just state "You are already at latest version" and not do anything, unless one does it via 
+
+----
+SELECT postgis_extensions_upgrade();
+----
+
+which would have SOLUTION 1.
+