Opened 12 years ago

Closed 8 years ago

Last modified 7 years ago

#4563 closed defect (fixed)

Python bindings installation ignores prefix

Reported by: Mike Taves Owned by: hobu
Priority: normal Milestone: 2.1.0
Component: PythonBindings Version: 1.11.2
Severity: normal Keywords:
Cc: Markus Neteler, antonio

Description

On a system where I don't have root privileges, and would like to install python bindings, I'm using:

configure --prefix=/some/path --with-python

where the prefix path I have group r/w privileges. However, I get an error with make install.

Here are excerpts of the error, where /some/src/gdal-1.9.0 is the source directory that I'm building from and /some/path is the desired install prefix:

$ make install
...
Libraries have been installed in:
   /some/path/lib
...
/bin/sh /some/src/gdal-1.9.0/libtool --mode=install /some/src/gdal-1.9.0/install-sh -c gdal-config-inst /some/path/bin/gdal-config
libtool: install: /some/src/gdal-1.9.0/install-sh -c gdal-config-inst /some/path/bin/gdal-config
make[1]: Leaving directory `/some/src/gdal-1.9.0/apps'
(cd swig; make install)
make[1]: Entering directory `/some/src/gdal-1.9.0/swig'
for dir in python ; do (cd $dir; make install) || exit; done
make[2]: Entering directory `/some/src/gdal-1.9.0/swig/python'
python setup.py install
running install
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/lib64/python2.6/site-packages/test-easy-install-12914.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/lib64/python2.6/site-packages/

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  http://peak.telecommunity.com/EasyInstall.html

Please make the appropriate changes for your system and try again.

make[2]: *** [install] Error 1
make[2]: Leaving directory `/some/src/gdal-1.9.0/swig/python'
make[1]: *** [install] Error 2
make[1]: Leaving directory `/some/src/gdal-1.9.0/swig'
make: *** [install-actions] Error 2

Normally, when I install python extensions on this machine, I use:

python setup.py install --prefix=/some/path

As a working hack solution, I modified swig/python/GNUmakefile to hardcode the prefix:

ifeq ($(PY_HAVE_SETUPTOOLS),1)
        $(PYTHON) setup.py install --prefix=/some/path
else
        $(PYTHON) setup.py install --prefix=$(DESTDIR)$(prefix)
endif

Is there a better way to do this? Is it safe to assume that the python bindings should have the same prefix? (i.e., gdal-config --prefix)

Change History (17)

comment:1 by Kurt Schwehr, 10 years ago

I am confused as to why the prefix should ignored if setuptools is installed. It looks like this behavior started at r12441. I believe prefix should be passed through if it has been set for the setuptools case.

comment:2 by Markus Neteler, 10 years ago

Cc: Markus Neteler added
Milestone: 1.11.0
Version: 1.9.01.10.1

Same issue on Scientific Linux (and I cannot manually modify Makefiles on a production system...).

Same issue reported in #3594

comment:3 by antonio, 10 years ago

Cc: antonio added

comment:4 by jeanjordaan, 10 years ago

This also breaks building gdal in a Python virtualenv.

comment:5 by Even Rouault, 9 years ago

Milestone: 1.11.0

comment:6 by Markus Neteler, 9 years ago

Milestone: 1.11.3
Version: 1.10.11.11.2

Any chance to get this solved?

comment:7 by Markus Neteler, 9 years ago

This hack works around the problem:

svn diff swig/python/GNUmakefile
Index: swig/python/GNUmakefile
===================================================================
--- swig/python/GNUmakefile	(revision 28527)
+++ swig/python/GNUmakefile	(working copy)
@@ -67,11 +67,11 @@
 	
 install:
 
-ifeq ($(PY_HAVE_SETUPTOOLS),1)
-	$(PYTHON) setup.py install 
-else
+#ifeq ($(PY_HAVE_SETUPTOOLS),1)
+#	$(PYTHON) setup.py install 
+#else
 	$(PYTHON) setup.py install --prefix=$(DESTDIR)$(prefix)
-endif
+#endif
 
 	for f in $(SCRIPTS) ; do $(INSTALL) ./scripts/$$f $(DESTDIR)$(INST_BIN) ; done

comment:8 by Even Rouault, 9 years ago

I've tried it, but it doesn't solve the problem for me. I get the following error

python setup.py install --prefix=/home/even/gdal/svn/trunk/dummyinstall
running install
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 2] No such file or directory: '/home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages/test-easy-install-6973.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages/

This directory does not currently exist.  Please create it and try again, or
choose a different installation directory (using the -d or --install-dir
option).

Indeed the /home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages directory doesn't exist ( /home/even/gdal/svn/trunk/dummyinstall was empty to do the test) , but I would have expected the install to do it...

If I create manually /home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages, then it fails with a different error message:

/usr/bin/python -E -c pass
TEST FAILED: /home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH

You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from.  The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages/

and your PYTHONPATH environment variable currently contains:

    ''

Here are some of your options for correcting the problem:

* You can choose a different installation directory, i.e., one that is
  on PYTHONPATH or supports .pth files

* You can add the installation directory to the PYTHONPATH environment
  variable.  (It must then also be on PYTHONPATH whenever you run
  Python and want to use the package(s) you are installing.)

* You can set up the installation directory to support ".pth" files by
  using one of the approaches described here:

  http://peak.telecommunity.com/EasyInstall.html#custom-installation-locations

Please make the appropriate changes for your system and try again.

And indeed if I define PYTHONPATH=/home/even/gdal/svn/trunk/dummyinstall/lib/python2.6/site-packages it succeeds...

comment:9 by antonio, 9 years ago

Possible solution for this issue available on github (https://github.com/OSGeo/gdal/pull/65)

Last edited 9 years ago by antonio (previous) (diff)

comment:10 by Even Rouault, 9 years ago

Milestone: 1.11.3

Unsetting milestone

comment:11 by antonio, 9 years ago

Milestone: 2.1.0
Resolution: fixed
Status: newclosed

Fixed in r30564. I assume that this fix will go in GDAL v2.1.0.

comment:12 by Even Rouault, 9 years ago

Resolution: fixed
Status: closedreopened

Antonio, this breaks the MacOsX Travis instance : https://travis-ci.org/rouault/gdal_coverage/builds/81242249 . Previously bindings were installed in /Library/Python/2.7/site-packages/ (https://travis-ci.org/rouault/gdal_coverage/builds/81193480), now they are installed in /usr/lib/python2.7/site-packages . I'm not sure why they are not found. The MacOSX Travis setup is there : https://github.com/rouault/gdal_coverage/blob/trunk_travis_macosx/.travis.yml

comment:13 by antonio, 9 years ago

Hi Even, I'm not an OSX expert but it seems to be that this issue is very specific of the system installation of python.

>>> sys.prefix
'/System/Library/Frameworks/Python.framework/Versions/2.7'
>>> distutils.sysconfig.get_python_lib()
'/Library/Python/2.7/site-packages'
>>> distutils.sysconfig.get_python_lib(prefix='/usr')
'/usr/lib/python2.7/site-packages'

If no prefix is specified the setup.py script uses the correct location for python packages. Anyway we used /usr as prefix so the setup script, consistently, installs the packages in '/usr/lib/python2.7/site-packages'.

I can implement a workaround for the "special case" in which prefix=/usr and we are using the system python but it probably would not be very elegant. Also I have still not investigated what happens if the --with-macosx-framework config option is used.

I will implement the workaround in my github fork so you can check it and let me know if you like it.

comment:14 by Even Rouault, 9 years ago

See feedback given by William in the mailing list.

comment:16 by Even Rouault, 8 years ago

Resolution: fixed
Status: reopenedclosed

Everything back to normal. Thanks !

comment:17 by strk, 7 years ago

The code is still not fully correct starting with python-3.4, see #6671

Note: See TracTickets for help on using tickets.