Opened 3 years ago

Closed 2 weeks ago

#2071 closed defect (fixed)

SciTools IRIS 2.10 missing from Python3

Reported by: darkblueb Owned by: osgeolive@…
Priority: major Milestone: OSGeoLive14.0
Component: OSGeoLive Keywords: python, IRIS, scitools
Cc: osgeolive@…

Description

the OSGeoLive version 12 based on Ubuntu Bionic (1804) currently installs Scitools IRIS via dot-deb packaging found here [0]

IRIS setup.py [1] "may be broken for Python3" in this environment per the packager Angelos Tzotsos. This ticket is a place to find a workaround or longer-term fix.

[0] https://launchpad.net/~gcpp-kalxas/+archive/ubuntu/iris

[1] https://github.com/SciTools/iris/blob/v2.1.x/setup.py

Change history (15)

comment:1 Changed 3 years ago by darkblueb

Phil Elson writes:

Thanks for letting us know. I’m sorry to say I can’t find the detail regarding the python 3 issue. 
If I were to guess though, I suspect the problem is that pyke forked itself and maintains two branches 
for its python 2 -> python 3 journey (don’t know why one would choose to do that). 

As a result, pyke for python 3 needs to come from a different source. 
To see how to work around that, the conda-forge build for this is done in 

https://github.com/conda-forge/pyke-feedstock/blob/master/recipe/meta.yaml#L12.

Hopefully my guess is on target, but if not, 
could you point me at the detail and I’ll try to help?
 
--

Philip Elson | Scientific Python Technical Lead
Analysis, Visualisation and Data Team (AVD)
Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United Kingdom
Email: philip.elson@metoffice.gov.uk  Website: www.metoffice.gov.uk
Our magazine Barometer is now available online at http://www.metoffice.gov.uk/barometer/

comment:2 Changed 3 years ago by darkblueb

Priority: majornormal

12dev beta2

defer until after RC 12.0

comment:3 Changed 2 years ago by kalxas

Milestone: OSGeoLive12.0OSGeoLive13.0

Ticket retargeted after milestone closed

comment:4 Changed 22 months ago by darkblueb

open in o13 alpha2

comment:5 Changed 21 months ago by kalxas

Thank you for the information that there is a separate source for python3.

I am now trying to make a python3-pyke package.

comment:6 Changed 21 months ago by kalxas

Iris 2.2.1 has landed in our ppa.

python3-pyke has also been created, but python3-iris fails to build.

comment:7 Changed 21 months ago by kalxas

Milestone: OSGeoLive13.0OSGeoLive14.0

comment:8 Changed 4 months ago by kalxas

During the OSGeo Virtual Code Sprint we decided: we will try to package py3 version (pike etc), else we will include as py2 in VM. e-mail to project to ask about 20.04

comment:9 Changed 4 months ago by kalxas

Cc: osgeolive@… added
Priority: normalcritical

comment:11 Changed 4 months ago by kalxas

Current Iris build error:

creating /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/000003000000.03.236.000128.1990.12.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/000003000000.03.236.004224.1990.12.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/000003000000.03.236.008320.1990.12.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/000003000000.16.202.000128.1860.09.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/001000000000.00.000.000000.1860.01.01.00.00.f.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/002000000000.44.101.131200.1920.09.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/008000000000.44.101.000128.1890.09.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/12187.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/HadCM2_ts_SAT_ann_18602100.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_level_lat_orig.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_lon_lat_press_orig.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_lon_lat_several.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_n10r13xy.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_time_press.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_tseries.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/abcza_pa19591997_daily_29.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/abcza_pa19591997_daily_29.b_1.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/abcza_pa19591997_daily_29.b_2.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/abxpa_press_lat.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/integer.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/model.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/ocean_xsect.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/st0fc699.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/st0fc942.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/st30211.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
creating /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_pp
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_pp/000003000000.03.236.000128.1990.12.01.00.00.b.pp.txt -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_pp
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_pp/000003000000.03.236.004224.1990.12.01.00.00.b.pp.txt -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_pp
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_pp/000003000000.03.236.008320.1990.12.01.00.00.b.pp.txt -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_pp
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_pp/000003000000.16.202.000128.1860.09.01.00.00.b.pp.txt -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_pp
creating /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/util
copying lib/iris/tests/results/util/as_compatible_shape_collapsed.cml -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/util
copying lib/iris/fileformats/_pyke_rules/fc_rules_cf.krb -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/fileformats/_pyke_rules
creating /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/unit/experimental/representation
copying lib/iris/tests/unit/experimental/representation/test_CubeRepresentation.py -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/unit/experimental/representation
/usr/bin/python3 tools/generate_std_names.py etc/cf-standard-name-table.xml /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/std_names.py
/usr/bin/python3 -c import os; from pyke.target_pkg import target_pkg; target_pkg.load = lambda *args, **kwargs: None; os.chdir(os.path.join('/<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build', 'iris', 'fileformats', '_pyke_rules')); from pyke import knowledge_engine; knowledge_engine.engine('')
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/config.py:84: UserWarning: Ignoring config item 'Resources':'test_data_dir' (section:option) as '/home/h05/itwl/projects/git/iris-test-data/test_data' is not a valid directory path.
  warnings.warn(msg.format(section, option, c_path))
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/fileformats/netcdf.py:1052: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if data_dtype.itemsize is 1:
Unable to create '/usr/lib/python3/dist-packages/pyke/krb_compiler/krbparser_tables.py'
[Errno 13] Permission denied: '/usr/lib/python3/dist-packages/pyke/krb_compiler/krbparser_tables.py'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/pyke/knowledge_engine.py", line 108, in __init__
    target_package.compile(self)
  File "/usr/lib/python3/dist-packages/pyke/target_pkg.py", line 285, in compile
    self.do_by_ext('compile',
  File "/usr/lib/python3/dist-packages/pyke/target_pkg.py", line 265, in do_by_ext
    return getattr(self, "%s_%s" % (prefix, ext))(filename, *args)
  File "/usr/lib/python3/dist-packages/pyke/target_pkg.py", line 298, in compile_krb
    return krb_compiler.compile_krb(rb_name, self.package_name,
  File "/usr/lib/python3/dist-packages/pyke/krb_compiler/__init__.py", line 114, in compile_krb
    ast = krbparser.parse(krbparser, filename)
  File "/usr/lib/python3/dist-packages/pyke/krb_compiler/krbparser.py", line 595, in parse
    scanner.init(scanner, debug, check_tables)
  File "/usr/lib/python3/dist-packages/pyke/krb_compiler/scanner.py", line 688, in init
    lexer = lex.lex(module=this_module, optimize=1,
  File "/usr/lib/python3/dist-packages/pyke/krb_compiler/ply/lex.py", line 1004, in lex
    lexobj.writetab(lextab,outputdir)
  File "/usr/lib/python3/dist-packages/pyke/krb_compiler/ply/lex.py", line 176, in writetab
    tf = open(filename,"w")
PermissionError: [Errno 13] Permission denied: '/usr/lib/python3/dist-packages/pyke/krb_compiler/scanner_tables.py'
error: command '/usr/bin/python3' failed with exit status 1
E: pybuild pybuild:341: build: plugin distutils failed with: exit code=1: /usr/bin/python3 setup.py build 
dh_auto_build: error: pybuild --build --test-nose -i python{version} -p 3.8 returned exit code 13
make: *** [debian/rules:8: build] Error 25
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
--------------------------------------------------------------------------------
Build finished at 2020-12-17T16:02:00Z

Last edited 3 months ago by kalxas (previous) (diff)

comment:12 Changed 3 months ago by kalxas

We need to reach out to IRIS team, perhaps they will have some feedback on the above error.

comment:13 Changed 3 months ago by kalxas

feedback from sebastic:

The build script is writing outside the build/install directory, it mustn't do that. The build is broken by design, because if an unpriviledged user builds the project it will fail as well. The iris people will need to help you to make it behave more appropriately.

comment:14 Changed 2 months ago by kalxas

Priority: criticalmajor

Based on the above feedback I propose we move Iris to VM only, where we can build it in a virtualenv

Note: See TracTickets for help on using tickets.