= RFC 24: GDAL Progressive Data Support = 

Author: Norman Barker[[BR]]
Contact: nbarker@ittvis.com[[BR]]
Status: Development

== Summary ==

To provide an interface for data streaming support to GDAL.  The RFC focuses on JPIP but should be generic to apply to other streaming / progressive formats.

== Definitions ==

'''JPIP''': JPEG 2000 Interactive Protocol

== Objective ==

To provide an interface to a streaming data source in a convenient manner for RasterIO operations within GDAL and to expose this interface via swig.

Note this RFC originally explored using an Observer pattern as per implementations in Java [http://java.sun.com/j2se/1.4.2/docs/api/javax/imageio/event/IIOReadUpdateListener.html imageio update] but after detailed discussions with [http://thread.gmane.org/gmane.comp.gis.gdal.devel/16409/focus=16630 Tamas Szekeres, Even Rouault and Adam Nowacki] a simpler interface is proposed to reduce the potential for problems by having the callback implemented by another thread.

Each swig wrapper language will have its own design pattern for event handling (the image updates) and the proposed interface will allow that pattern implementation to be provided, even if simulated.

== Implementation ==

The implementation is a definition of an interface for streaming support data support within GDAL. Concrete implementations of this interface will follow.  Currently the most convenient JPIP streaming developer library is [http://www.kakadusoftware.com Kakadu] however since GDAL is also developer library, only stubs can be distributed to conform to Kakadu licensing ([http://trac.osgeo.org/gdal/wiki/JP2KAK JP2KAK]).  Commercial vendors are also interested in using GDAL for streaming support and a standard interface will allow these commercial plugins to be incorporated. e.g. [http://www.gdal.org/frmt_jp2ecw.html ECW], [http://www.gdal.org/frmt_mrsid.html MrSID].

To support reading streaming data from a progressive server it will be necessary to communicate between different threads.  In general it's safer not to do this by shared memory, but by explicit message passing using asynchronous queues. e.g. [http://library.gnome.org/devel/glib/2.12/glib-Asynchronous-Queues.html Glib Asynchronous Queues]