g.gui.animation: parallel rendering with d.rast and d.vect

[huhabla]

I would suggest to use d.rast and d.vect to create the render images in parallel in g.gui.animation.

At the moment r.out.ppm is used to create the render images which is IMHO insufficient. There is no way to support vector maps with this approach.

In addition, the size of the render images should not depend on the current region settings. It should be dependent on the size of the animation render window. This can not be archived with the current approach without tweaking the region settings.

Using d.rast and d.vect lets specify the size of the render images using GRASS_WIDTH and GRASS_HEIGHT in the render process and allows the parallel rendering of several images at the same time.

I have attached a patch to enable the parallel rendering of raster and vector maps in the BitmapProvider class. It seems to me that this approach is much faster on a multicore system then the current approach.

[huhabla]

Patch to enable d.rast and d.vect in g.gui.animation

[huhabla]

I have implemented a patch that enables the visualization of space time vector datasets with g.gui.animation using d.vect. The patch is attached.

[huhabla]

This patch enables the visualization of space time vector datasets

[huhabla]

Simple script to test the g.gui.animation command line arguments

[huhabla]

Anna, many thanks for applying the patch. I noticed that the rendering engine can be chosen in the main GUI. I am curious where the switch to change the number of parallel render processes is located? :)

I was thinking about how to speedup the rendering process a bit more:

For now only the rendering is done in parallel. The in parallel created png images are read in the main process and converted into wxImages that are put into the image buffer. This was necessary because wx objects can not be serialized by the multiprocess module and stored in the queue object, only the file names are stored.

A better solution would be to read the png images using wxImage in the rendering process, convert it into a raw image (PNM, PGM or pixmaps) and store it into the queue object as a pure python binary array. The main process would put this raw image into the image buffer. Hence png reading, decoding and conversion into a raw image format will be performed in parallel as well.

In case that image overlay should be supported (vector map over raster map, legends, text ...) i would suggest the following approach:

The functionality of g.cairocomp and g.pnmcomp should be put into a C overlay library that can be accessed in python via ctypes. It is important to remove the fatal error calls in the overlay library and replace them with an intelligent error handling that does not call exit. A single rendering process may call d.rast, d.vect, d.legend, d.text and so on several times. The created png images can then be read (wxImage?) and overlaid using the C overlay library functionality via ctypes. The resulting raw binary image will then transferred into a binary python array that can be stored into the queue object.

This overlay library can be used in the wxGUI as well, to overlay several maps. Using external grass modules to do the overlay job is not very efficient i guess?

[annakrat]

Replying to huhabla:

Anna, many thanks for applying the patch. I noticed that the rendering engine can be chosen in the main GUI. I am curious where the switch to change the number of parallel render processes is located? :)

Parallel rendering is not implemented in the main gui (maybe there should be?) so it should be in the animation settings dialog (which is not existing yet:)).

I was thinking about how to speedup the rendering process a bit more:

For now only the rendering is done in parallel. The in parallel created png images are read in the main process and converted into wxImages that are put into the image buffer. This was necessary because wx objects can not be serialized by the multiprocess module and stored in the queue object, only the file names are stored.

A better solution would be to read the png images using wxImage in the rendering process, convert it into a raw image (PNM, PGM or pixmaps) and store it into the queue object as a pure python binary array. The main process would put this raw image into the image buffer. Hence png reading, decoding and conversion into a raw image format will be performed in parallel as well.

Have you tried something like this? I have tried it with wx.Image.GetData() and SetData(), however it seems that the queue gets full and the application hangs. Have you any idea how to make it work?

In case that image overlay should be supported (vector map over raster map, legends, text ...) i would suggest the following approach:

The functionality of g.cairocomp and g.pnmcomp should be put into a C overlay library that can be accessed in python via ctypes. It is important to remove the fatal error calls in the overlay library and replace them with an intelligent error handling that does not call exit. A single rendering process may call d.rast, d.vect, d.legend, d.text and so on several times. The created png images can then be read (wxImage?) and overlaid using the C overlay library functionality via ctypes. The resulting raw binary image will then transferred into a binary python array that can be stored into the queue object.

This overlay library can be used in the wxGUI as well, to overlay several maps. Using external grass modules to do the overlay job is not very efficient i guess?

I tested overlaying images (PNG) in wxPython (using wx.MemoryDC) and it's working however it seems to be even slower than the current approach, I think mainly because of slow PNG compression. Current approach uses combination of .ppm and .pgm (mask) files. To overlay these ppm images with wxPython is fast, however we need to load the mask (wx.Mask) (from pgm file) and set it which slows it down. So I have currently no better solution using pure wxPython. I am not sure if using PIL and numpy would help.

Putting g.pnmcomp to library would probably help but I am not sure if we can save significant time. Of course, it would be nice to have everything in library with no G_fatal calls but this is a different discussion.

[annakrat]

The original enhancement is done, the discussion about g.pnmcomp is more complete in #2033.