Opened 7 years ago

Closed 5 years ago

Last modified 5 years ago

#1678 closed defect (fixed)

[raster] GDAL_DATA dependency

Reported by: robe Owned by: pracine
Priority: medium Milestone: PostGIS 2.2.0
Component: raster Version: trunk
Keywords: history Cc: mateusz@…

Description

Perhaps I'm the only one affected by this GDAL_DATA environment dependency. I'm mostly affected as a package manager.

The issue I see is that if I set this environment variable during installer process, I could adversely affect other applications users are running. Worst yet, and uninstall of one postgis version or postgis or another postgis could cause PostGIS or the other apps to stop working.

This is very undesirable of course and worst yet is a really hard issue for someone to debug. IS there some way we can internalize this dependency similar to what we do with proj so that I can installed

gdal-data folder in the postgis-2.0 folder and ensure uninstallation or uninstallation of some other app doesn't cause these troubling problems?

Change History (24)

comment:1 Changed 7 years ago by pracine

Summary: raster: GDAL_DATA dependency[raster] GDAL_DATA dependency

Is this only on Windows?

comment:2 Changed 7 years ago by Bborie Park

Technically, this issue could be on any platform. As GDAL's data directory is determined upon compilation, you can find out what your GDAL installation's path is

gdal-config --datadir

If the output of the above is NOT where your GDAL files are installed, you'll need to set GDAL_DATA.

comment:3 Changed 7 years ago by Bborie Park

By the looks of it, the GDAL_DATA path can be set in C. We could add a flag to configure.ac to specify GDAL_DATA (and possibly PROJSO).

http://trac.osgeo.org/gdal/wiki/ConfigOptions

I know what strk is going to say. This could also be done as a GUC.

comment:4 Changed 7 years ago by robe

It doesn't really help much to set it at configure time since I don't know where people will have PostgreSQL installed.

ideally it should just look for gdal_data in the postgis-2.0/gdal-data folder first and if not present then fall back on the environment variable.

comment:5 Changed 7 years ago by pramsey

Does it have to be an absolute path...? (Fearing that it does...)

comment:6 Changed 7 years ago by Bborie Park

I think it does. Otherwise, the question becomes "relative to what?".

comment:7 Changed 7 years ago by robe

Well how do we set the proj directory? That one checks relative to postgis-2.0 folder or postgis folder. Have to confirm that's working.

comment:8 Changed 7 years ago by pramsey

If we know where it is relative to "us" (and perhaps Proj can give guidance there) we can set the GDAL data directory with

CPLSetConfigOption("GDAL_DATA", pathtoit);

comment:9 Changed 7 years ago by Bborie Park

How should the path of GDAL_DATA be specified by the builder/user so that CPLSetConfigOption can be used? In configure? A GUC? Some config file?

comment:10 Changed 7 years ago by Bborie Park

Search the file-system as I'm assuming the output from "gdal-config --datadir" is incorrect?

comment:11 Changed 7 years ago by pramsey

Assuming most linux builds will use a system GDAL and will therefore pick up GDAL_DATA automatically we're left really with packagers who need to override that behavior (windows). So maybe we can just hard-code a "../data" path in where desired. So an entry in postgis_config.h.in that, when non-empty, is used to set GDAL_DATA (wow, hacktastic) and we could add in a build-time configure option to force it later if the first level hack works. A moderately better approach would be to do it somewhere in libpgcommon, checking a PostgreSQL system variable (don't ask me...) that provides information about the install/run paths that we can leverage off of.

comment:12 Changed 7 years ago by mloskot

Cc: mateusz@… added

comment:13 Changed 6 years ago by mloskot

I just grabbed the latest PostGIS 2.0.1 installer for Windows and PostgreSQL 9.2.

IMHO, setting GDAL_DATA by the installer is a not a idea.

Couldn't PostGIS recover the location in run-time, on Windows, and set it with GDAL's CPLSetConfigOption according to scheme like this:

  • postgis-2.0.dll has DllMain? defined
  • DllMain? calls GetModuleFileName? to find its own location, e.g.
    C:\Program Files (x86)\PostgreSQL\9.2\lib\postgis-2.0.dll
    
  • Assuming both, postgis-2.0.dll and lib\ are static parts, it's easy to find root folder:
    C:\Program Files (x86)\PostgreSQL\9.2
    
  • Assuming both, gdal-data\ directory name is static part, it's easy to compile GDAL_DATA location:
    C:\Program Files (x86)\PostgreSQL\9.2\gdal-data\
    
  • And, stores it in non-exported DLL global:
    static char* gPostGISGDALDataFolder = NULL;
    
  • Finally, each PostgreSQL sets its own copy of environment:
    CPLSetConfigOption( "GDAL_DATA", gPostGISGDALDataFolder);
    

comment:14 Changed 6 years ago by Bborie Park

Milestone: PostGIS 2.1.0PostGIS Future

comment:15 Changed 6 years ago by Bborie Park

Milestone: PostGIS FuturePostGIS 2.2.0

Since SFCGAL took the plunge with custom GUCs, it'd be nice to have a GUC for GDAL_DATA. Granted, it'd be nice if the PSC made a decision if I am permitted to do this...

comment:16 Changed 6 years ago by robe

+1 from me.

comment:17 Changed 6 years ago by Bborie Park

Resolution: fixed
Status: newclosed

Added in r11554

comment:18 Changed 6 years ago by Bborie Park

Keywords: history added

comment:19 Changed 5 years ago by edechaux

Resolution: fixed
Status: closedreopened

I have tested this fix on postigs 2.1.0 and postgresql 9.2.4.

While using the parameter postgis.gdal.datapath from a pqsql session works, the double dot in the name prevents it from being set in the postgresql.conf file. Upon pg startup, the following error is shown :

LOG:  syntax error in file "/home/edechaux/test_postgis/postgresql.conf" line 574, near token "postgis.gdal.datapath"
FATAL:  configuration file "/home/edechaux/test_postgis/postgresql.conf" contains errors

I suggest the parameter name to be changed to postgis.gdal_datapath. The pg server starts up without warning/error and the parameter is still working properly :

edechaux=# select current_setting('postgis.gdal_datapath');
           current_setting
-------------------------------------
 /appli/pgsql/idicsi_2.0/share/gdal/
(1 row)

edechaux=# select postgis_full_version();
                                                                             postgis_full_version
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 POSTGIS="2.1.0 r11822" GEOS="3.4.2-CAPI-1.8.2 r3921" PROJ="Rel. 4.8.0, 6 March 2012" GDAL="GDAL 1.10.1, released 2013/08/26" LIBXML="2.7.6" LIBJSON="UNKNOWN" TOPOLOGY RASTER
(1 row)

comment:20 Changed 5 years ago by Bborie Park

Resolution: fixed
Status: reopenedclosed

This enhancement was added to -trunk (will be PostGIS 2.2). It does not exist in 2.1

comment:21 Changed 5 years ago by edechaux

I know but backporting to 2.1 is very easy and postgis version not significant here.

The issue is that postgresql parser does not handle two dots in a GUC parameter. From postgresql 9.2.4 sources in src/backend/utils/misc/guc-file.l :

                /* first token on line is option name */
                if (token != GUC_ID && token != GUC_QUALIFIED_ID)
                        goto parse_error;

and

LETTER                  [A-Za-z_\200-\377]
LETTER_OR_DIGIT [A-Za-z_0-9\200-\377]

ID                              {LETTER}{LETTER_OR_DIGIT}*
QUALIFIED_ID    {ID}"."{ID}

[...]

{ID}                    return GUC_ID;
{QUALIFIED_ID}  return GUC_QUALIFIED_ID;

So the only dot a custom parameter should use is the one spliting the module identifier and name of the parameter, as seen here.

Please reconsider reopening this ticket.

comment:22 Changed 5 years ago by Bborie Park

So this problem only happens for setting the value in postgresql.conf? Within a session, this has no issues (due to behavior exhibited in the regression tests).

comment:23 in reply to:  22 Changed 5 years ago by edechaux

Replying to dustymugs:

So this problem only happens for setting the value in postgresql.conf? Within a session, this has no issues (due to behavior exhibited in the regression tests).

Yes, allowing the parameter to be used in the configuration file will allow any application to use postgis/gdal by default, without needing to set a session parameter.

comment:24 Changed 5 years ago by Bborie Park

In that case, please file a new ticket.

Note: See TracTickets for help on using tickets.