Opened 6 years ago

Closed 3 years ago

#2071 closed defect (fixed)

SciTools IRIS 2.10 missing from Python3

Reported by: darkblueb Owned by: osgeolive@…
Priority: major Milestone: OSGeoLive14.0
Component: OSGeoLive Keywords: python, IRIS, scitools
Cc: osgeolive@…

Description

the OSGeoLive version 12 based on Ubuntu Bionic (1804) currently installs Scitools IRIS via dot-deb packaging found here [0]

IRIS setup.py [1] "may be broken for Python3" in this environment per the packager Angelos Tzotsos. This ticket is a place to find a workaround or longer-term fix.

[0] https://launchpad.net/~gcpp-kalxas/+archive/ubuntu/iris

[1] https://github.com/SciTools/iris/blob/v2.1.x/setup.py

Change history (15)

comment:1 by darkblueb, 6 years ago

Phil Elson writes:

Thanks for letting us know. I’m sorry to say I can’t find the detail regarding the python 3 issue. 
If I were to guess though, I suspect the problem is that pyke forked itself and maintains two branches 
for its python 2 -> python 3 journey (don’t know why one would choose to do that). 

As a result, pyke for python 3 needs to come from a different source. 
To see how to work around that, the conda-forge build for this is done in 

https://github.com/conda-forge/pyke-feedstock/blob/master/recipe/meta.yaml#L12.

Hopefully my guess is on target, but if not, 
could you point me at the detail and I’ll try to help?
 
--

Philip Elson | Scientific Python Technical Lead
Analysis, Visualisation and Data Team (AVD)
Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United Kingdom
Email: philip.elson@metoffice.gov.uk  Website: www.metoffice.gov.uk
Our magazine Barometer is now available online at http://www.metoffice.gov.uk/barometer/

comment:2 by darkblueb, 6 years ago

Priority: majornormal

12dev beta2

defer until after RC 12.0

comment:3 by kalxas, 5 years ago

Milestone: OSGeoLive12.0OSGeoLive13.0

Ticket retargeted after milestone closed

comment:4 by darkblueb, 5 years ago

open in o13 alpha2

comment:5 by kalxas, 5 years ago

Thank you for the information that there is a separate source for python3.

I am now trying to make a python3-pyke package.

comment:6 by kalxas, 5 years ago

Iris 2.2.1 has landed in our ppa.

python3-pyke has also been created, but python3-iris fails to build.

comment:7 by kalxas, 5 years ago

Milestone: OSGeoLive13.0OSGeoLive14.0

comment:8 by kalxas, 3 years ago

During the OSGeo Virtual Code Sprint we decided: we will try to package py3 version (pike etc), else we will include as py2 in VM. e-mail to project to ask about 20.04

comment:9 by kalxas, 3 years ago

Cc: osgeolive@… added
Priority: normalcritical

comment:11 by kalxas, 3 years ago

Current Iris build error:

creating /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/000003000000.03.236.000128.1990.12.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/000003000000.03.236.004224.1990.12.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/000003000000.03.236.008320.1990.12.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/000003000000.16.202.000128.1860.09.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/001000000000.00.000.000000.1860.01.01.00.00.f.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/002000000000.44.101.131200.1920.09.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/008000000000.44.101.000128.1890.09.01.00.00.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/12187.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/HadCM2_ts_SAT_ann_18602100.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_level_lat_orig.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_lon_lat_press_orig.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_lon_lat_several.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_n10r13xy.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_time_press.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/aaxzc_tseries.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/abcza_pa19591997_daily_29.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/abcza_pa19591997_daily_29.b_1.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/abcza_pa19591997_daily_29.b_2.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/abxpa_press_lat.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/integer.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/model.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/ocean_xsect.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/st0fc699.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/st0fc942.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf/st30211.b_0.cdl -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_netcdf
creating /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_pp
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_pp/000003000000.03.236.000128.1990.12.01.00.00.b.pp.txt -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_pp
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_pp/000003000000.03.236.004224.1990.12.01.00.00.b.pp.txt -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_pp
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_pp/000003000000.03.236.008320.1990.12.01.00.00.b.pp.txt -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_pp
copying lib/iris/tests/results/usecases/pp_to_cf_conversion/to_pp/000003000000.16.202.000128.1860.09.01.00.00.b.pp.txt -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/usecases/pp_to_cf_conversion/to_pp
creating /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/util
copying lib/iris/tests/results/util/as_compatible_shape_collapsed.cml -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/results/util
copying lib/iris/fileformats/_pyke_rules/fc_rules_cf.krb -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/fileformats/_pyke_rules
creating /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/unit/experimental/representation
copying lib/iris/tests/unit/experimental/representation/test_CubeRepresentation.py -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/tests/unit/experimental/representation
/usr/bin/python3 tools/generate_std_names.py etc/cf-standard-name-table.xml /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/std_names.py
/usr/bin/python3 -c import os; from pyke.target_pkg import target_pkg; target_pkg.load = lambda *args, **kwargs: None; os.chdir(os.path.join('/<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build', 'iris', 'fileformats', '_pyke_rules')); from pyke import knowledge_engine; knowledge_engine.engine('')
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/config.py:84: UserWarning: Ignoring config item 'Resources':'test_data_dir' (section:option) as '/home/h05/itwl/projects/git/iris-test-data/test_data' is not a valid directory path.
  warnings.warn(msg.format(section, option, c_path))
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_scitools-iris/build/iris/fileformats/netcdf.py:1052: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if data_dtype.itemsize is 1:
Unable to create '/usr/lib/python3/dist-packages/pyke/krb_compiler/krbparser_tables.py'
[Errno 13] Permission denied: '/usr/lib/python3/dist-packages/pyke/krb_compiler/krbparser_tables.py'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/pyke/knowledge_engine.py", line 108, in __init__
    target_package.compile(self)
  File "/usr/lib/python3/dist-packages/pyke/target_pkg.py", line 285, in compile
    self.do_by_ext('compile',
  File "/usr/lib/python3/dist-packages/pyke/target_pkg.py", line 265, in do_by_ext
    return getattr(self, "%s_%s" % (prefix, ext))(filename, *args)
  File "/usr/lib/python3/dist-packages/pyke/target_pkg.py", line 298, in compile_krb
    return krb_compiler.compile_krb(rb_name, self.package_name,
  File "/usr/lib/python3/dist-packages/pyke/krb_compiler/__init__.py", line 114, in compile_krb
    ast = krbparser.parse(krbparser, filename)
  File "/usr/lib/python3/dist-packages/pyke/krb_compiler/krbparser.py", line 595, in parse
    scanner.init(scanner, debug, check_tables)
  File "/usr/lib/python3/dist-packages/pyke/krb_compiler/scanner.py", line 688, in init
    lexer = lex.lex(module=this_module, optimize=1,
  File "/usr/lib/python3/dist-packages/pyke/krb_compiler/ply/lex.py", line 1004, in lex
    lexobj.writetab(lextab,outputdir)
  File "/usr/lib/python3/dist-packages/pyke/krb_compiler/ply/lex.py", line 176, in writetab
    tf = open(filename,"w")
PermissionError: [Errno 13] Permission denied: '/usr/lib/python3/dist-packages/pyke/krb_compiler/scanner_tables.py'
error: command '/usr/bin/python3' failed with exit status 1
E: pybuild pybuild:341: build: plugin distutils failed with: exit code=1: /usr/bin/python3 setup.py build 
dh_auto_build: error: pybuild --build --test-nose -i python{version} -p 3.8 returned exit code 13
make: *** [debian/rules:8: build] Error 25
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
--------------------------------------------------------------------------------
Build finished at 2020-12-17T16:02:00Z

Last edited 3 years ago by kalxas (previous) (diff)

comment:12 by kalxas, 3 years ago

We need to reach out to IRIS team, perhaps they will have some feedback on the above error.

comment:13 by kalxas, 3 years ago

feedback from sebastic:

The build script is writing outside the build/install directory, it mustn't do that. The build is broken by design, because if an unpriviledged user builds the project it will fail as well. The iris people will need to help you to make it behave more appropriately.

comment:14 by kalxas, 3 years ago

Priority: criticalmajor

Based on the above feedback I propose we move Iris to VM only, where we can build it in a virtualenv

Note: See TracTickets for help on using tickets.