Opened 5 years ago

Closed 20 months ago

Last modified 8 months ago

#4563 closed defect (fixed)

Python bindings installation ignores prefix

Reported by: Mike Toews Owned by: hobu
Priority: normal Milestone: 2.1.0
Component: PythonBindings Version: 1.11.2
Severity: normal Keywords:
Cc: neteler, antonio

Description

On a system where I don't have root privileges, and would like to install python bindings, I'm using:

configure --prefix=/some/path --with-python

where the prefix path I have group r/w privileges. However, I get an error with make install.

Here are excerpts of the error, where /some/src/gdal-1.9.0 is the source directory that I'm building from and /some/path is the desired install prefix:

$ make install
...
Libraries have been installed in:
   /some/path/lib
...
/bin/sh /some/src/gdal-1.9.0/libtool --mode=install /some/src/gdal-1.9.0/install-sh -c gdal-config-inst /some/path/bin/gdal-config
libtool: install: /some/src/gdal-1.9.0/install-sh -c gdal-config-inst /some/path/bin/gdal-config
make[1]: Leaving directory `/some/src/gdal-1.9.0/apps'
(cd swig; make install)
make[1]: Entering directory `/some/src/gdal-1.9.0/swig'
for dir in python ; do (cd $dir; make install) || exit; done
make[2]: Entering directory `/some/src/gdal-1.9.0/swig/python'
python setup.py install
running install
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/lib64/python2.6/site-packages/test-easy-install-12914.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/lib64/python2.6/site-packages/

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  http://peak.telecommunity.com/EasyInstall.html

Please make the appropriate changes for your system and try again.

make[2]: *** [install] Error 1
make[2]: Leaving directory `/some/src/gdal-1.9.0/swig/python'
make[1]: *** [install] Error 2
make[1]: Leaving directory `/some/src/gdal-1.9.0/swig'
make: *** [install-actions] Error 2

Normally, when I install python extensions on this machine, I use:

python setup.py install --prefix=/some/path

As a working hack solution, I modified swig/python/GNUmakefile to hardcode the prefix:

ifeq ($(PY_HAVE_SETUPTOOLS),1)
        $(PYTHON) setup.py install --prefix=/some/path
else
        $(PYTHON) setup.py install --prefix=$(DESTDIR)$(prefix)
endif

Is there a better way to do this? Is it safe to assume that the python bindings should have the same prefix? (i.e., gdal-config --prefix)

Change History (17)

comment:1 Changed 4 years ago by Kurt Schwehr

I am confused as to why the prefix should ignored if setuptools is installed. It looks like this behavior started at r12441. I believe prefix should be passed through if it has been set for the setuptools case.

comment:2 Changed 3 years ago by neteler

Cc: neteler added
Milestone: 1.11.0
Version: 1.9.01.10.1

Same issue on Scientific Linux (and I cannot manually modify Makefiles on a production system...).

Same issue reported in #3594

comment:3 Changed 3 years ago by antonio

Cc: antonio added

comment:4 Changed 3 years ago by jeanjordaan

This also breaks building gdal in a Python virtualenv.

comment:5 Changed 2 years ago by Even Rouault

Milestone: 1.11.0

comment:6 Changed 2 years ago by neteler

Milestone: 1.11.3
Version: 1.10.11.11.2

Any chance to get this solved?

comment:7 Changed 2 years ago by neteler

This hack works around the problem:

svn diff swig/python/GNUmakefile
Index: swig/python/GNUmakefile
===================================================================
--- swig/python/GNUmakefile	(revision 28527)
+++ swig/python/GNUmakefile	(working copy)
@@ -67,11 +67,11 @@
 	
 install:
 
-ifeq ($(PY_HAVE_SETUPTOOLS),1)
-	$(PYTHON) setup.py install 
-else
+#ifeq ($(PY_HAVE_SETUPTOOLS),1)
+#	$(PYTHON) setup.py install 
+#else
 	$(PYTHON) setup.py install --prefix=$(DESTDIR)$(prefix)
-endif
+#endif
 
 	for f in $(SCRIPTS) ; do $(INSTALL) ./scripts/$$f $(DESTDIR)$(INST_BIN) ; done

comment:8 Changed 2 years ago by Even Rouault

I've tried it, but it doesn't solve the problem for me. I get the following error

python setup.py install --prefix=/home/even/gdal/svn/trunk/dummyinstall
running install
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 2] No such file or directory: '/home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages/test-easy-install-6973.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages/

This directory does not currently exist.  Please create it and try again, or
choose a different installation directory (using the -d or --install-dir
option).

Indeed the /home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages directory doesn't exist ( /home/even/gdal/svn/trunk/dummyinstall was empty to do the test) , but I would have expected the install to do it...

If I create manually /home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages, then it fails with a different error message:

/usr/bin/python -E -c pass
TEST FAILED: /home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH

You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from.  The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages/

and your PYTHONPATH environment variable currently contains:

    ''

Here are some of your options for correcting the problem:

* You can choose a different installation directory, i.e., one that is
  on PYTHONPATH or supports .pth files

* You can add the installation directory to the PYTHONPATH environment
  variable.  (It must then also be on PYTHONPATH whenever you run
  Python and want to use the package(s) you are installing.)

* You can set up the installation directory to support ".pth" files by
  using one of the approaches described here:

  http://peak.telecommunity.com/EasyInstall.html#custom-installation-locations

Please make the appropriate changes for your system and try again.

And indeed if I define PYTHONPATH=/home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages it succeeds...

comment:9 Changed 21 months ago by antonio

Possible solution for this issue available on github (https://github.com/OSGeo/gdal/pull/65)

Last edited 21 months ago by antonio (previous) (diff)

comment:10 Changed 21 months ago by Even Rouault

Milestone: 1.11.3

Unsetting milestone

comment:11 Changed 20 months ago by antonio

Milestone: 2.1.0
Resolution: fixed
Status: newclosed

Fixed in r30564. I assume that this fix will go in GDAL v2.1.0.

comment:12 Changed 20 months ago by Even Rouault

Resolution: fixed
Status: closedreopened

Antonio, this breaks the MacOsX Travis instance : https://travis-ci.org/rouault/gdal_coverage/builds/81242249 . Previously bindings were installed in /Library/Python?/2.7/site-packages/ (https://travis-ci.org/rouault/gdal_coverage/builds/81193480), now they are installed in /usr/lib/python2.7/site-packages . I'm not sure why they are not found. The MacOSX Travis setup is there : https://github.com/rouault/gdal_coverage/blob/trunk_travis_macosx/.travis.yml

comment:13 Changed 20 months ago by antonio

Hi Even, I'm not an OSX expert but it seems to be that this issue is very specific of the system installation of python.

>>> sys.prefix
'/System/Library/Frameworks/Python.framework/Versions/2.7'
>>> distutils.sysconfig.get_python_lib()
'/Library/Python/2.7/site-packages'
>>> distutils.sysconfig.get_python_lib(prefix='/usr')
'/usr/lib/python2.7/site-packages'

If no prefix is specified the setup.py script uses the correct location for python packages. Anyway we used /usr as prefix so the setup script, consistently, installs the packages in '/usr/lib/python2.7/site-packages'.

I can implement a workaround for the "special case" in which prefix=/usr and we are using the system python but it probably would not be very elegant. Also I have still not investigated what happens if the --with-macosx-framework config option is used.

I will implement the workaround in my github fork so you can check it and let me know if you like it.

comment:14 Changed 20 months ago by Even Rouault

See feedback given by William in the mailing list.

comment:16 Changed 20 months ago by Even Rouault

Resolution: fixed
Status: reopenedclosed

Everything back to normal. Thanks !

comment:17 Changed 8 months ago by strk

The code is still not fully correct starting with python-3.4, see #6671

Note: See TracTickets for help on using tickets.