Changes between Version 49 and Version 50 of rfc24_progressive_data_support


Ignore:
Timestamp:
Sep 2, 2008, 4:08:54 PM (16 years ago)
Author:
normanb
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • rfc24_progressive_data_support

    v49 v50  
    1717To provide an interface to a streaming data source in a convenient manner for RasterIO operations within GDAL and to expose this interface via swig.
    1818
    19 Note this RFC originally explored using an Observer pattern as per implementations in Java [http://java.sun.com/j2se/1.4.2/docs/api/javax/imageio/event/IIOReadUpdateListener.html imageio update].  The RFC then also discussed the Asynchronous Message Queue pattern, but after detailed discussions with [http://thread.gmane.org/gmane.comp.gis.gdal.devel/16409/focus=16630 Tamas Szekeres, Even Rouault, Adam Nowacki and Frank Warmerdam] an even simpler interface is proposed which wraps the more unpleasant developer tasks of operating with the streaming protocol but allows the user greater flexibility in interacting with the stream in their preferred language.  This is implemented by using a context object to act as an intermediary between the user layer and the format driver.
     19Note this RFC originally explored using an Observer pattern as per implementations in Java [http://java.sun.com/j2se/1.4.2/docs/api/javax/imageio/event/IIOReadUpdateListener.html imageio update].  The RFC then also discussed the [http://www.gtk.org/api/2.6/glib/glib-Asynchronous-Queues.html Asynchronous Message Queue pattern], but after detailed discussions with [http://thread.gmane.org/gmane.comp.gis.gdal.devel/16409/focus=16630 Tamas Szekeres, Even Rouault, Adam Nowacki and Frank Warmerdam] an even simpler interface is proposed which wraps the more unpleasant developer tasks of operating with the streaming protocol but allows the user greater flexibility in interacting with the stream in their preferred language.  This is implemented by using a context object to act as an intermediary between the user layer and the format driver.  The approach recommended in the RFC is a half-way abstraction of the protocol as it requires some knowledge that this is a streaming format and how to optimise the driver from the user interface.  This is an acceptable, and actually desirable trade-off, since it allows the swig wrappers to implement high speed data access / display in a pattern suitable for the language.
     20 
    2021
    2122== Implementation ==
    … …  
    2324The implementation is a definition of an interface for streaming support data support within GDAL. Concrete implementations of this interface will follow.  Currently the most convenient JPIP streaming developer library is [http://www.kakadusoftware.com Kakadu] however since GDAL is also a developer library, only stubs can be distributed to conform to Kakadu licensing ([http://trac.osgeo.org/gdal/wiki/JP2KAK JP2KAK]).  Vendors are also interested in using GDAL for streaming support and a standard interface will allow these plugins to be incorporated. e.g. [http://www.gdal.org/frmt_jp2ecw.html ECW], [http://www.gdal.org/frmt_mrsid.html MrSID].
    2425
    25 To support reading streaming data from a progressive server it will be necessary to communicate between different threads.  In general it's safer not to do this by shared memory, but by explicit message passing using asynchronous queues. e.g. [http://library.gnome.org/devel/glib/2.12/glib-Asynchronous-Queues.html Glib Asynchronous Queues], [http://en.wikipedia.org/wiki/Message_queue Wikipedia Message Queues]
    2626
    27 This interface results in a subclass of GDALDataset, and currently supports dataset level access only, not raster band access.  Access to the stream per band is not precluded by this design and can be added as a later RFC.
    28 
    29 === Class Diagram ===
    30 
    31 [[Image(class.png)]]
    32 
    33 Implementing GDALAsyncDataset allows the user application to decide when and how often the updates occur, and reduces the potential negative impacts of threading by providing a clear synchonization point, NextAsyncRasterIOMessage().
    34 
    35 GDALAsyncRasterIOMessage contains a buffer that is described the getParameters() function which returns a struct with the parameters passed into ProgressiveRasterIO.  Note that no lock() / unlock() on the buffer is required as each buffer represents a call to the server and is a separate instance.
    36 
    37 Each GDALAsyncRasterIOMessage will be small, it only contains the data for the currently requested window at a particular resolution level. These requests are usually proportional, i.e small window implies high resolution, large window implies low resolution.
    38 
    39 === Sequence of Events ===
    40 
    41 [[Image(sequence.png)]]
    42 
    43   1. GDALOpen calls Open(Xxx) on all registered drivers with a jpip:// parameter and the first driver that returns a response is the format driver for that user.
    44   1. Streaming image servers are usually session based so that the server knows which data has already been sent to the client.
    45   1. ProgressiveRasterIO requests a region of the image from the driver. ''In the specific case of JPIP, the dimensions of the input buffer, bufXSize, bufYSize and the region Xoff, yOff, xSize, ySize are used to calculate the best resolution level and the corresponding fsiz and rsiz parameters to the server.  i.e. bufXSize and bufYSize are the desirable screen size, and the region size is at the base resolution of the image.''
    46   1. Check whether we need to stop making requests to the server.
    47   1. In response to the ProgressiveRasterIO call the format driver makes an asynchronous request to the server for data matching the requested resolution and window.
    48   1. The driver parses the response from the server and sets the buffer on GDALAsyncRasterIOMessage.
    49   1. GDALAsyncRasterIOMessage object is added to the format driver internal data queue.  If the queue size limit is exceeded the last message drops off.  Note the data for this message is still retained in the wavelet cache for the driver - just the rendering is discarded.
    50   1. Check there is a data message to render.
    51   1. Retrieve the next data message and render / process.
    52   1. Finished with the data message, release the resource associated with it.
    53   1. Free the message data buffer.
     27== Pseudo-Code ==
    5428
    5529
     30