Opened 2 years ago

Closed 2 years ago

#2352 closed defect (fixed)

mapnik python demo ol15

Reported by: darkblueb Owned by: osgeolive@…
Priority: critical Milestone: OSGeoLive15.0
Component: OSGeoLive Keywords: mapnik, jupyter
Cc: osgeolive@…

Description (last modified by darkblueb)

a mapnik python demo included in the dot-deb packaging fails [0]

earlier #osgeolive versions ref: https://trac.osgeo.org/osgeolive/ticket/2255

mapnik will render to a Jupyter Notebook:

https://git.osgeo.org/gitea/darkblueb/ol14_ipynb_misc/src/branch/master/mapnik-py

TODO demo for ol15

==

mapnik

user@ol15nightly:~$ cp -R /usr/share/doc/python3-mapnik/examples/* mapnik_test/


user@ol15nightly:~/mapnik_test/demo/python$ python3 rundemo.py 
Traceback (most recent call last):
  File "/home/user/mapnik_test/demo/python/rundemo.py", line 322, in <module>
    mapnik.render(m, im)
RuntimeError: Cannot initialize proj_transform for given projections without proj4 support (-DMAPNIK_USE_PROJ4): '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs'->'+proj=lcc +ellps=GRS80 +lat_0=49 +lon_0=-95 +lat+1=49 +lat_2=77 +datum=NAD83 +units=m +no_defs'

Attachments (1)

Screenshot_20220803_203821.png (21.5 KB ) - added by kalxas 2 years ago.

Download all attachments as: .zip

Change History (23)

comment:1 by darkblueb, 2 years ago

Description: modified (diff)
Keywords: mapnik jupyter added
Milestone: OSGeoLive15.0

comment:2 by darkblueb, 2 years ago

Description: modified (diff)

comment:3 by kalxas, 2 years ago

Cc: osgeolive@… added
Priority: normalcritical

comment:4 by kalxas, 2 years ago

Quickstart is also failing when using the MapProxy to render mapnik tiles.

comment:6 by kalxas, 2 years ago

Type: taskdefect

comment:7 by darkblueb, 2 years ago

build 162 ol15

user@osgeolive:~/Downloads/python-mapnik-master/test$ ipython3
Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.31.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import mapnik

In [2]: m = mapnik.Map(512, 512)
   ...: 

In [3]: mapnik.load_map(m, 'data/good_maps/wgs842merc_reprojection.xml')
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-3-26cbb3ea7887> in <module>
----> 1 mapnik.load_map(m, 'data/good_maps/wgs842merc_reprojection.xml')

RuntimeError: Cannot initialize projection 'epsg:3857 ' without proj4 support (-DMAPNIK_USE_PROJ4) in Map at line 1 of 'data/good_maps/wgs842merc_reprojection.xml'

comment:8 by sanak, 2 years ago

The error message itself seems to come from mapnik core library. https://github.com/mapnik/mapnik/blob/v3.1.x/src/proj_transform.cpp#L129-L134

#ifdef MAPNIK_USE_PROJ4
            source_.init_proj4();
            dest_.init_proj4();
#else
            throw std::runtime_error(std::string("Cannot initialize proj_transform for given projections without proj4 support (-DMAPNIK_USE_PROJ4): '") + source_.params() + "'->'" + dest_.params() + "'");
#endif

I compared libmapnik between OSGeoLive 14.0 and 15.0beta1 environments, and I noticed that OSGeoLive 15.0beta1 libmapnik3.1 seems not to link to libproj.

  • OSGeoLive 14.0
    $ dpkg -l | grep libmapnik
    ii  libmapnik3.0 ...
    :
    $ dpkg -L libmapnik3.0
    :
    /usr/lib/libmapnik.so.3.0
    $ ldd /usr/lib/libmapnik.so.3.0 | grep libproj
            libproj.so.19 ...
    
  • OSGeoLive 15.0beta1
    $ dpkg -l | grep libmapnik
    ii  libmapnik3.1 ...
    :
    $ dpkg -L libmapnik3.1
    :
    /usr/lib/libmapnik.so.3.1
    $ ldd /usr/lib/libmapnik.so.3.1 | grep libproj
    

So, I guess that rebuilding libmapnik3.1 with -DMAPNIK_USE_PROJ4 flag or downgrading to libmapnik3.0 may solve this issue.

comment:9 by Bas Couwenberg, 2 years ago

Mapnik 3.x doesn't support proj.h: https://github.com/mapnik/mapnik/issues/4036

I cannot be bothered to backport the changes like Fedora did, see:

Last edited 2 years ago by Bas Couwenberg (previous) (diff)

in reply to:  9 comment:10 by sanak, 2 years ago

Replying to Bas Couwenberg:

Mapnik 3.x doesn't support proj.h: https://github.com/mapnik/mapnik/issues/4036

I cannot be bothered to backport the changes like Fedora did, see:

Okay, thanks for the information.

From the Fedora person (Tom Hughes)'s patch's comment link (https://github.com/mapnik/mapnik/pull/4202) and his (@tomhughes) fork fedora branch's history, applying the following commit may be enough, but I don't know well about Debian packaging things...

OSGeoLive 15 rc1 documentation freeze for the english version is planned at tomorrow (or a day after tomorrow), so I think that there is no enough time to search alternative way instead of using mapnik.

comment:11 by darkblueb, 2 years ago

I have the Mapnik core cloned here, and can manually apply that patch to the code. What happens after that?

Mapnik loads a dozen major graphics libraries. The build is not easy, but it is possible. For language bindings, I can load, test and understand the results of python, but probably not NPM or the others so quickly.

in reply to:  12 comment:13 by sanak, 2 years ago

Replying to kalxas:

Great!
Thanks for the update!

by kalxas, 2 years ago

comment:15 by kalxas, 2 years ago

After installing the new packages on top of build163, the test code seems to work now (see screenshot) but the mapproxy demo still fails. Perhaps a configuration issue?

comment:16 by darkblueb, 2 years ago

confirm build 167-staging

user@osgeolive:~$ ldd /usr/lib/libmapnik.so.3.1.0 | grep proj
        libproj.so.22 => /lib/x86_64-linux-gnu/libproj.so.22 (0x00007f7cf20b4000)

user@osgeolive:~/mapnik-ex0/demo/python$ python3 rundemo.py 

 15 maps have been rendered in the current directory:
- demo.png
- demo256.png
- demo64_binary_transparency.png
- demo128_colors_hextree_no_alpha.png
- demo_high.jpg
- demo_low.jpg
- demo.tif
- demo.webp
- demo_med.webp
- demo_low.webp
- demo.pdf
- demo.ps
- demo.svg
- demo_cairo_rgb.png
- demo_cairo_argb.png

Have a look!

user@osgeolive:~/mapnik-ex0/demo/python$ file *
README.txt:                          ASCII text
demo.pdf:                            PDF document, version 1.5, 1 pages
demo.png:                            PNG image data, 800 x 600, 8-bit/color RGBA, non-interlaced
demo.ps:                             PostScript document text conforming DSC level 3.0, Level 2
demo.svg:                            SVG Scalable Vector Graphics image
demo.tif:                            TIFF image data, little-endian, direntries=14, height=600, bps=22912, compression=deflate, PhotometricIntepretation=RGB, width=800
demo.webp:                           RIFF (little-endian) data, Web/P image, VP8 encoding, 800x600, Scaling: [none]x[none], YUV color, decoders should clamp
demo128_colors_hextree_no_alpha.png: PNG image data, 800 x 600, 8-bit colormap, non-interlaced
demo256.png:                         PNG image data, 800 x 600, 8-bit colormap, non-interlaced
demo64_binary_transparency.png:      PNG image data, 800 x 600, 8-bit colormap, non-interlaced
demo_cairo_argb32.png:               PNG image data, 800 x 600, 8-bit/color RGB, non-interlaced
demo_cairo_rgb24.png:                PNG image data, 800 x 600, 8-bit/color RGB, non-interlaced
demo_high.jpg:                       JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision 8, 800x600, components 3
demo_highest.webp:                   RIFF (little-endian) data, Web/P image, VP8 encoding, 800x600, Scaling: [none]x[none], YUV color, decoders should clamp
demo_low.jpg:                        JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision 8, 800x600, components 3
demo_low.webp:                       RIFF (little-endian) data, Web/P image, VP8 encoding, 800x600, Scaling: [none]x[none], YUV color, decoders should clamp
map.xml:                             XML 1.0 document, Unicode text, UTF-8 text
rundemo.py:                          Python script, Unicode text, UTF-8 text executable
user@osgeolive:~/mapnik-data$ ipython3
Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.31.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import mapnik
In [2]: m = mapnik.Map(512,512)
In [3]: mapnik.load_map(m, 'good_maps/wgs842merc_reprojection.xml')
In [4]: m
Out[4]: <mapnik._mapnik.Map at 0x7fca3cf58770>
Last edited 2 years ago by darkblueb (previous) (diff)

comment:17 by sanak, 2 years ago

Okay, thanks for the information.
I also confirmed it.

user@osgeolive:~$ ldd /usr/lib/libmapnik.so.3.1.0 | grep proj
        libproj.so.22 => /lib/x86_64-linux-gnu/libproj.so.22 (0x00007f1252217000)

but the mapproxy demo still fails. Perhaps a configuration issue?

About this mapproxy demo failing, the error message was changed to failed to initialize projection with: '+init=epsg:3857', and from googling a bit, the cause seems to be in mapproxy side.

From above 2nd link, I patched /usr/lib/python3/dist-packages/mapproxy/source/mapnik.py as follows locally, then I could show the mapnik layer correctly.

-         m.srs = '+init=%s' % str(query.srs.srs_code.lower())
+         m.srs = str(query.srs.srs_code.lower())

in reply to:  14 comment:18 by sanak, 2 years ago

About above patching location, I guess that patching python3-mapproxy itself may be better than patching it on OSGeoLive bin/install_mapproxy.sh.

Last edited 2 years ago by sanak (previous) (diff)

comment:20 by kalxas, 2 years ago

Thanks all for the feedback!

I have now ported mapproxy 1.15.1 from debian and patched the package itself. We will know the result in a while.

comment:21 by kalxas, 2 years ago

The mapproxy mapnik demo now works!

I will build a new iso and close this after we confirm.

comment:22 by kalxas, 2 years ago

Resolution: fixed
Status: newclosed

Confirmed fixed in build178

Note: See TracTickets for help on using tickets.