pyarrow 0.0.0-1 doesn't work and makes both geopandas and pandas not working as well

[andreaerdna]

Refs: ​https://github.com/qgis/QGIS/issues/59776#issuecomment-2524744848

with both arrow-cpp 17.0.0-1 and python3-pyarrow 0.0.0-1 installed

import pyarrow as pa
days = pa.array([1, 12, 17, 23, 28], type=pa.int8())
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "pyarrow\\array.pxi", line 273, in pyarrow.lib.array
  File "pyarrow\\array.pxi", line 55, in pyarrow.lib._is_array_like
  File "pyarrow\\pandas-shim.pxi", line 124, in pyarrow.lib._PandasAPIShim._have_pandas_internal
  File "pyarrow\\pandas-shim.pxi", line 103, in pyarrow.lib._PandasAPIShim._check_import
  File "pyarrow\\pandas-shim.pxi", line 106, in pyarrow.lib._PandasAPIShim._check_import
  File "pyarrow\\pandas-shim.pxi", line 50, in pyarrow.lib._PandasAPIShim._import_pandas
  File "C:\OS022F~1\apps\Python312\Lib\site-packages\pandas\__init__.py", line 26, in <module>
    from pandas.compat import (
  File "C:\OS022F~1/apps/qgis-dev/./python\qgis\utils.py", line 966, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OS022F~1\apps\Python312\Lib\site-packages\pandas\compat\__init__.py", line 27, in <module>
    from pandas.compat.pyarrow import (
  File "C:\OS022F~1/apps/qgis-dev/./python\qgis\utils.py", line 966, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OS022F~1\apps\Python312\Lib\site-packages\pandas\compat\pyarrow.py", line 10, in <module>
    _palv = Version(Version(pa.__version__).base_version)
                    ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OS022F~1\apps\Python312\Lib\site-packages\pandas\util\version\__init__.py", line 341, in __init__
    match = self._regex.search(version)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'NoneType'

The presence of python3-pyarrow 0.0.0-1 installed, makes impossible to import both geopandas and pandas:

import geopandas as gpd
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\OS022F~1/apps/qgis-dev/./python\qgis\utils.py", line 966, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OS022F~1\apps\Python312\Lib\site-packages\geopandas\__init__.py", line 3, in <module>
    from geopandas.geoseries import GeoSeries
  File "C:\OS022F~1/apps/qgis-dev/./python\qgis\utils.py", line 966, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OS022F~1\apps\Python312\Lib\site-packages\geopandas\geoseries.py", line 9, in <module>
    import pandas as pd
  File "C:\OS022F~1/apps/qgis-dev/./python\qgis\utils.py", line 966, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OS022F~1\apps\Python312\Lib\site-packages\pandas\__init__.py", line 26, in <module>
    from pandas.compat import (
  File "C:\OS022F~1/apps/qgis-dev/./python\qgis\utils.py", line 966, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OS022F~1\apps\Python312\Lib\site-packages\pandas\compat\__init__.py", line 27, in <module>
    from pandas.compat.pyarrow import (
  File "C:\OS022F~1/apps/qgis-dev/./python\qgis\utils.py", line 966, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OS022F~1\apps\Python312\Lib\site-packages\pandas\compat\pyarrow.py", line 10, in <module>
    _palv = Version(Version(pa.__version__).base_version)
                    ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OS022F~1\apps\Python312\Lib\site-packages\pandas\util\version\__init__.py", line 341, in __init__
    match = self._regex.search(version)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'NoneType'

[andreaerdna]

pyarrow, geopandas and pandas worked well in previous QGIS version like QGIS LTR 3.34.9 when both arrow-cpp and python3-pyarrow OSGeo4W packages were at version 15.0.2-1.

[andreaerdna]

The issue is due to the fact that, in python3-pyarrow 0.0.0-1, pyarrow.__version__ = None, which in turn is due to the fact that the "_generated_version" module is missing.

A simple workaround, which also makes both geopandas and pandas work well when python3-pyarrow 0.0.0-1 is installed, is to change line 59 in apps\Python312\Lib\site-packages\pyarrow\__init__.py from __version__ = None to __version__ = '17.0.0'.

[andreaerdna]

It looks like this issue also make unusable external plugins that relies on the pandas and geopandas modules, like the featured plugin Data Plotly. See ​https://github.com/ghtmtt/DataPlotly/issues/363.

[andreaerdna]

It looks like the issue has been fixed with python3-pyarrow 17.0.0-1 after ​https://github.com/jef-n/OSGeo4W/commit/cfc6b5f4084cf373a90d5766a6f54f37eceed894.