Opened 10 years ago

Closed 10 years ago

#282 closed defect (worksforme)

python path getting overwritten

Reported by: lpinner Owned by: maphew
Priority: minor Component: Package
Version: Keywords: python
Cc: warmerdam

Description

In %OSGEO4W_ROOT%\bin\o4w_env.bat the \etc\ini\*.bat files are called in a loop (alphabetically i presume). When %OSGEO4W_ROOT%\etc\ini\orfeotoolbox-python.bat gets called, it sets PYTHONPATH to %OSGEO4W_ROOT%\apps\orfeotoolbox\python. This gets overwritten when %OSGEO4W_ROOT%\etc\ini\python.bat is called afterwards i.e. SET PYTHONPATH=%OSGEO4W_ROOT%\apps\Python27.

This causes QGIS to report an error on startup ("Could not find the Python bindings for OrfeoToolbox?, which are required to run the this module") and breaks the python bindings in the osgeo4w shell generally.

Perhaps these batch files should set PYTHONPATH by appending/prepending, i.e. for etc\ini\python.bat:

SET PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\Python27

or

IF "%PYTHONPATH%"=="" (
    SET PYTHONPATH=%OSGEO4W_ROOT%\apps\Python27
) ELSE (
    SET PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\Python27
)

Change History (8)

comment:1 in reply to:  description Changed 10 years ago by lpinner

Replying to lpinner:

and breaks the python bindings in the osgeo4w shell generally.

Oops, I meant "and breaks the OrfeoToolbox? python bindings in the osgeo4w shell generally" not all python bindings

comment:2 Changed 10 years ago by warmerdam

Cc: warmerdam added
Keywords: python added
Owner: changed from osgeo4w-dev@… to maphew

Sounds plausible. We need to respect that multiple packages may need to extend PYTHONPATH. +maphew who I think is the python packager now.

comment:3 Changed 10 years ago by maphew

Status: newassigned

I don't know that we can rely/assume the call order for the batch files is alphabetical.

I'll use the second example for now. Long term I'd rather intelligently only add path items which don't exist already (here and in pkg-shell), something that appears to be frustratingly hard on Windows.

Thanks for the report.

comment:4 in reply to:  3 Changed 10 years ago by lpinner

Replying to maphew:

I don't know that we can rely/assume the call order for the batch files is alphabetical.

I don't think the order is important, I only mentioned it as it was the OTB pythonpath that was getting overwritten, not the other way around. Should probably do something similar for all batch files that set environment variables that might also be set/reset/extended by another batch file.

I'll use the second example for now. Long term I'd rather intelligently only add path items which don't exist already (here and in pkg-shell), something that appears to be frustratingly hard on Windows.

It's kinda ugly, but what about something like:

rem Couldn't figure out out to get the output of findstr
rem into a variable, so use errorlevel instead
rem Idea from http://stackoverflow.com/questions/2634959/2-batch-string-questions

set testpythonpath=%OSGEO4W_ROOT%\apps\Python27
echo %PYTHONPATH%|findstr /i "%testpythonpath%">nul 
IF  %errorlevel%==0 goto EXIT 
IF "%PYTHONPATH%"=="" (
    SET PYTHONPATH=%testpythonpath% 
) ELSE (
    SET PYTHONPATH=%PYTHONPATH%;%testpythonpath%
)
:EXIT

comment:5 Changed 10 years ago by maphew

Thanks Luke. That lead, eventually, to a complete treatment here: http://stackoverflow.com/questions/141344/how-to-check-if-directory-exists-in-path, along with a very good description of what makes this a difficult issue in the first place.

perhaps something like this (untested, about to run out of battery power)

:pypath
  set _path=%path%
  set PATH=%PYTHONPATH%
  call b:\bin\addpath %OSGEO4W_ROOT%\apps\Python27
  set PYTHONPATH=%PATH%
  set PATH=%_path%
  goto :eof

comment:6 Changed 10 years ago by maphew

This pattern is successful, after replacing the call to findstr in addpath with an absolute path (%windir%\system32\findstr.exe):

set _save=%path%
set path=%pythonpath%
set _add=%OSGEO4W_ROOT%\apps\orfeotoolbox\python

call addpath _add

set path=%_save%
set _save=
set _add=

Wordy though. I'm tempted to try and re-work addpath to allow a parameter of variable-name or raw string.

In any case the python package doesn't set PYTHONPATH, only PYTHONHOME, so the only thing I/we can do is chart out a course for package maintainers to follow.

comment:7 in reply to:  6 Changed 10 years ago by lpinner

Replying to maphew:

In any case the python package doesn't set PYTHONPATH, only PYTHONHOME, so the only thing I/we can do is chart out a course for package maintainers to follow.

My bad. My %OSGEO4W_ROOT%\etc\ini\python.bat sets both PYTHONHOME and PYTHONPATH. I must have modified it manually at some stage, though I don't remember doing so. I do know I have had issues due to multiple GDAL&Python installs, so maybe that's why I modified the .bat...? Apologies for the noise!

comment:8 Changed 10 years ago by maphew

Resolution: worksforme
Status: assignedclosed

no worries about noise. It added enough itch to find a solution to more general PATH problems that has been bugging me for awhile (now in pkg-shell v 1.0.0-9).

I'm resolving this as "worksforme". Please re-open if it continues to be an issue.

Note: See TracTickets for help on using tickets.