| 10 | | With the help of GRASS Python modules, It easy to pull chunks from raster data as Numpy array. A new rendering model (in-memory) can be added to wxGUI which can directly display a numpy array as `wx.Image`. This will obviously increase the performance of rendering layers because the overhead of reading and writing data to file (PNG adds a compression during packing and unpacking) can be saved. Also the extent based rendering will allow to read portion of raster image than whole dataset. Also a tile based technique can be added to this which improves the rendering performance further. |
| | 19 | Related tickets: |
| | 20 | * #1719 GRASS 7 Monitor command line support |
| | 21 | * #1926 g.gui.animation: parallel rendering with d.rast and d.vect |
| | 22 | |
| | 23 | Related discussion on mailing list: |
| | 24 | * [http://lists.osgeo.org/pipermail/grass-dev/2012-August/059099.html Speed map display in wxGUI on bigger monitors], ([http://osgeo-org.1560.x6.nabble.com/Speed-map-display-in-wxGUI-on-bigger-monitors-td4992997.html at Nabble]) |
| | 25 | |
| | 26 | Related changesets: |
| | 27 | |
| | 28 | * r54465 fix for not updating statusbar (possibly slowed down the rendering) |
| | 29 | * r52436 wxGUI: delete unused wx.Image instances when rendering |
| | 30 | |
| | 31 | |
| | 32 | == General problems affecting current wxGUI == |
| | 33 | |
| | 34 | * it is in Python and simple loop is always slower in Python then in C/C++ |
| | 35 | * loading of dynamic (wxWidgets, wxPython) libs and modules always takes some time, when you open display/GUI for the first time (when I'm opening the second display it is much faster) |
| | 36 | * rendering to files and then displaying this files on screen is slower than direct drawing of data to screen |
| | 37 | |
| | 38 | About the last point, we are using rendering to files because displaying/drawing of maps is done by modules, not library functions (the functionality is in the library). Moreover, it is not safe to call GRASS library functions from GUI since they call `G_fatal_error()` which calls `exit()`, so on error the whole GUI ends (without an error because stderr is written into GUI console which ends too). |
| | 39 | |
| | 40 | == Currently used options == |
| | 41 | |
| | 42 | * wxGUI renders to uncompressed PPM and uses `g.pnmcomp` |
| | 43 | * wxWidgets-based `xganim` reads a raster file directly and stores data in `wxImage` |
| | 44 | * it is fast in C++ but to do the same in Python you have to use some !NumPy magic to make it at least comparably fast (the reading and displaying data is done in loops) |
| | 45 | * `G_fatal_error()` and `exit()` are still here |
| | 46 | * in order to provide full drawing capabilities code from `d.*` modules would have to be moved to library |
| | 47 | * wxPython-based `g.gui.animation` uses `d.*` with process Queue |
| | 48 | * `d.*` commands enables to render both vector and raster |
| | 49 | * process Queue makes it faster (on multicore) even if the images are written to disk |
| | 50 | * older wxPython-based `g.gui.animation` used `r.out.ppm` and !NumPy |
| | 51 | * NumPy syntax is unreadable in this case |
| | 52 | * Python loop was just so slow to be used |
| | 53 | * wxNVIZ uses OpenGL |
| | 54 | * TODO: provide basic info here |
| | 55 | |
| | 56 | == Proposal: Do the composition in the wxGUI == |
| | 57 | |
| | 58 | The wxPython GUI should really be doing its own compositing rather than relying upon `g.pnmcomp`. |
| | 59 | |
| | 60 | The original reason for composition by a module was that Tk doesn't support PNG. That isn't relevant to the wxPython GUI. |
| | 61 | |
| | 62 | Can PIL be used for this? It is a new dependency only partially since currently Cartographic Composer preview depends on it. |
| | 63 | |
| | 64 | (glynn, http://lists.osgeo.org/pipermail/grass-dev/2012-August/059188.html, with some comments) |
| | 65 | |
| | 66 | == Proposal: Use compressed files to create smaller files == |
| | 67 | |
| | 68 | change the PPM files to something compressed (see ML) |
| | 69 | |
| | 70 | http://lists.osgeo.org/pipermail/grass-dev/2012-August/059099.html |
| | 71 | |
| | 72 | Then I got impression that there is some issue with PNG driver (if we want to use it instead of PPM) and also that cairo driver could help but it is not ready. |
| | 73 | |
| | 74 | |
| | 75 | == Experimental code: PNG rendering and composing in Python == |
| | 76 | |
| | 77 | Experimental code to test PNG rendering and composing in Python: attachment:png_rendering_pil_composition.diff:ticket:1719 |
| | 78 | |
| | 79 | In the diff you can see some changes which are necessary for rendering PNG (instead of PPM) and composing of images in Python using PIL (instead of `g.pnmcomp`). |
| | 80 | |
| | 81 | It is not completely working code. It is only experiment to see if this can be faster. |
| | 82 | |
| | 83 | The answer is no. I don't have any numbers but user experience is completely the same. |
| | 84 | |
| | 85 | Files are much smaller and there is one file less (thanks to composing in Python) but during zooming/panning the most of the time is spent with disk IO (tested on Ubuntu 10.04). Zooming and panning require re-executing the `d.*` commands to generate new images. |
| | 86 | |
| | 87 | The downside of the compressed format is that the `d.*` command will take longer as it has to compress the output file. |
| | 88 | |
| | 89 | If desired, we could have a single file without the compression overhead by using the PNG writer from lib/pngdriver, which allows the compression level to be set via the `GRASS_PNG_COMPRESSION` environment variable. (''Move this paragraph to some proposal?'') |
| | 90 | |
| | 91 | ''(wenzeslaus, annakrat, comment:14:ticked:1719, 2012-09-07, glynn comments included)'' |
| | 92 | |
| | 93 | == Proposal: Output binary data from `d.*` modules == |
| | 94 | |
| | 95 | We can output binary data to stdout and read them in Python directly. This would avoid disk IO. In other words stdout instead would replace usage of a file. |
| | 96 | |
| | 97 | That requires either storing all layers in memory, regardless of whether or not they are displayed, or re-generating layers if they are disabled then re-enabled. It also eliminates the possibility of implementing a decent caching mechanism (i.e. being able to undo zoom/pan operations by re-using the previous images rather than having to re-generate them). |
| | 98 | |
| | 99 | And using pipes may be slower than disk (if the Python side is using `select()`, there will be a context switch for each pipe-buffer-size of data). |
| | 100 | |
| | 101 | ''(wenzeslaus, comment:14:ticked:1719, 2012-09-07, glynn comments included)'' |
| | 102 | |
| | 103 | |
| | 104 | |
| | 105 | == Proposal: Use X Pixmaps == |
| | 106 | |
| | 107 | In theory, the fastest solution should be to use the Cairo driver with output to X Pixmaps. The `d.*` modules generate output in video memory, and may be hardware accelerated. Compositing occurs entirely in video memory, and may be hardware accelerated. The main unknowns are how hard wxWidgets makes this, and whether something similar can be achieved on other platforms. |
| | 108 | |
| | 109 | === Details and disadvantages === |
| | 110 | |
| | 111 | The cairo driver can render to an X Pixmap. These can then be composited via `g.cairocomp`. Rendering and compositing may be hardware-accelerated, and the resulting raster data will never leave video memory. |
| | 112 | |
| | 113 | However: |
| | 114 | |
| | 115 | * Requires X. I don't think that there's any equivalent on Windows. X is only reliably available on Linux. Not really an option for Windows or Mac. |
| | 116 | * It relies upon the fact that an X client can create resources (e.g. Pixmaps) on the server which are retained after the program terminates. |
| | 117 | * Requires that cairo was built with XRender support (although that's almost certain on X; cairo was created largely to facilitate the use of XRender). |
| | 118 | * Can wxWidgets use an X Pixmap directly (i.e. without copying the contents to client memory and back again)? |
| | 119 | * Leaving "stale" images in the X server's memory is even worse than leaving stale PPM files in the temp directory. Even keeping all of the "live" images in the X server's memory may be an issue on some systems. |
| | 120 | |
| | 121 | === wxPython implementation === |
| | 122 | |
| | 123 | In wxWidgets there is wxBitmap ([http://wxpython.org/docs/api/wx.Bitmap-class.html wxPython], [http://wxpython.org/Phoenix/docs/html/Bitmap.html wxPython Phoenix], [http://docs.wxwidgets.org/2.8/wx_wxbitmap.html wxWidgets]). This wxBitmap can be directly drawn on some wx widget/window. This operation is fast. The data for an X Pixmap resides within the X server. On the client side, a Pixmap is just an XID (a 32-bit integer identifying a server-side resource). For X11-based versions of wxWidgets, a wxBitmap probably has an associated Pixmap, but I can't see any way to get at it, or to create a wxBitmap for an existing Pixmap. |
| | 124 | |
| | 125 | === Rendering of the image === |
| | 126 | |
| | 127 | If `GRASS_PNGFILE` ends in "`.xid`", the cairo driver will use the XRender back-end, and will write the XID of the underlying Pixmap to the file specified by $GRASS_PNGFILE. It will also call `XSetCloseDownMode(dpy, RetainTemporary)`, which results in server-side resource being retained when the X connection is closed. |
| | 128 | |
| | 129 | Another program can read the XID from this file, and use it with any Xlib function expecting a Pixmap or Drawable argument. E.g. `g.cairocomp` uses `cairo_xlib_surface_create_with_xrender_format()`, which takes the XID of an existing Drawable as an argument ([http://www.cairographics.org/manual/cairo-XLib-XRender-Backend.html#cairo-xlib-surface-create-with-xrender-format link to doc]). |
| | 130 | |
| | 131 | Thus, the `d.*` module creates the Pixmap on the X server and leaves it there for other programs to use. |
| | 132 | |
| | 133 | ''(glynn, comment:11:ticked:1719, 2012-09-05, with some other comments)'' |
| | 134 | |
| | 135 | |
| | 136 | == Proposal: Use different rendering engine for GUI and for monitors == |
| | 137 | |
| | 138 | Use different rendering engine for the main (big) GUI and for (wx) monitors. This engine could use library functions and can call `exit()` because it would end only one monitor, not the whole GUI (and the error output is written into the command line). |
| | 139 | |
| | 140 | But there are two problems. The first is mirroring of display architecture provided by modules and the second is maintenance of additional code (but this can be only minor issue if it is well designed). Basically it would be necessary to move all display (rendering) code into the library and use this library for `d.*` modules and wx monitors. However, it is not clear if ctypes would be enough to implement it in wxPython and thus (possibly) part of the code should be in wxWidgets (which could be easier with wxPython Phoenix). |
| | 141 | |
| | 142 | The idea of two different approaches for main GUI and monitors is not new. The current wxGUI (at least before r55230) uses different approach for layer rendering (the difference is the number of help files and the opacity handling). |
| | 143 | |
| | 144 | ''(Sep 4, 2012, #1719)'' |
| | 145 | |
| | 146 | == Proposal: Directly access data with !NumPy == |
| | 147 | |
| | 148 | With the help of GRASS Python modules, it easy to pull chunks from raster data as !NumPy array. A new rendering model (in-memory) can be added to wxGUI which can directly display a !NumPy array as `wx.Image`. This will obviously increase the performance of rendering layers because the overhead of reading and writing data to file (PNG adds a compression during packing and unpacking) can be saved. Also the extent based rendering will allow to read portion of raster image than whole dataset. Also a tile based technique can be added to this which improves the rendering performance further. |
