Broken venv created with python-qgis.bat

[nicogodet]

I spent some times searching for the best way to create an environnement for VSCode with proper linting, etc...

Best way I found is to create a venv using python-qgis.bat.

C:\OSGeo4W\bin\python-qgis.bat -m venv --system-site-packages .venv

It requieres few extra steps (won't describe here as irrelevant).

The issue is inside the .venv\pyvenv.cfg file created. According to ​PEP 405, this file is read and used to set a bunch of variables including PYTHONHOME.

Content of created file:

home = C:\OSGeo4W\bin
include-system-site-packages = true
version = 3.12.6
executable = C:\OSGeo4W\bin\python.exe
command = C:\OSGeo4W\bin\python.exe -m venv --system-site-packages D:\Desktop\temp\.venv

Activate venv and start python.

>.venv\Scripts\activate.bat

(.venv) D:\Desktop\temp>python
Could not find platform independent libraries <prefix>
Python 3.12.6 (main, Sep  7 2024, 18:59:42) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

Notice this line: Could not find platform independent libraries <prefix>

Deactivate and alter .venv\pyvenv.cfg:

home = C:\OSGeo4W\apps\Python312
include-system-site-packages = true
version = 3.12.6
executable = C:\OSGeo4W\apps\Python312\python.exe
command = C:\OSGeo4W\apps\Python312\python.exe -m venv --system-site-packages D:\Desktop\temp\.venv

(.venv) D:\Desktop\temp>deactivate

[...] Alter .venv\pyvenv.cfg

D:\Desktop\temp>.venv\Scripts\activate.bat

(.venv) D:\Desktop\temp>python
Python 3.12.6 (main, Sep  7 2024, 18:59:42) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

See, no more Could not find platform independent libraries <prefix>.

The root cause is that python-qgis.bat call python exe which is in C:\OSGeo4W\bin instead of calling the right exe in C:\OSGeo4W\apps\Python312 where Lib, libs, include, Scripts, ... folders are.

[akominlsfi]

Saw a familiar ticket title, we've had similar issues with our plugin development on Windows machines. AFAIK to make qgis python work with venvs, it requires some level of patching either to the installation directly or to the venv.

We use only one ltr/non-ltr branch of qgis per osgeo4w install directory, so our patching is like this now:

• copy some dlls from install/bin to install/apps/python3xx/dlls (for python to find the stdlib ssl/crypto/sqlite/lzma dlls directly)
• create .pth with os.add_dll_directory(install/bin) (to make python find other possible site-package dlls)
• create .pth with os.add_dll_directory(install/apps/qt5/bin) (to make python find qt dlls)
• create .pth with os.add_dll_directory(install/apps/qgis/bin) & path for install/apps/qgis/python (to make python find qgis dlls and qgis imports)

This allows us to use the install/apps/python3xx/python.exe directly, it has "import qgis" & "import PyQt5" etc usable, and venvs created with --system-site-packages also work in the same way.

Another option would be this tool, that does similar patching to the created venvs:

​https://github.com/GispoCoding/qgis-venv-creator

Previous issues related to this same thing with some more details:

​https://trac.osgeo.org/osgeo4w/ticket/814 (external zlib dll issue that crashed the python.exe on launch was fixed)

​https://trac.osgeo.org/osgeo4w/ticket/692