RFC 24: GDAL Progressive Data Support

Author: Norman Barker
Contact: nbarker@…
Status: Development

Summary

Provide an interface for data streaming support in GDAL. The RFC focuses on JPIP, but should be generic enough to apply to other streaming / progressive formats.

Definitions

JPIP: JPEG 2000 Interactive Protocol

Objective

To provide an interface to a streaming data source in a convenient manner for RasterIO operations within GDAL and to expose this interface via swig.

Note this RFC originally explored using an Observer pattern as per implementations in Java ​imageio update. The RFC then also discussed the ​Asynchronous Message Queue pattern, but after detailed discussions with ​Tamas Szekeres, Even Rouault, Adam Nowacki and Frank Warmerdam an even simpler interface is proposed which wraps the more unpleasant developer tasks of operating with the streaming protocol but allows the user greater flexibility in interacting with the stream in their preferred language. This is implemented by using a context object to act as an intermediary between the user layer and the format driver. The approach recommended in the RFC is a half-way abstraction of the protocol as it requires some knowledge that this is a streaming format and how to optimise the driver from the user interface. This is an acceptable, and actually desirable trade-off, since it allows the swig wrappers to implement high speed data access / display in a pattern suitable for the language.

Implementation

The implementation is a definition of an interface for streaming support data support within GDAL. Concrete implementations of this interface will follow. Currently the most convenient JPIP streaming developer library is ​Kakadu however since GDAL is also a developer library, only stubs can be distributed to conform to Kakadu licensing (JP2KAK). Vendors are also interested in using GDAL for streaming support and a standard interface will allow these plugins to be incorporated. e.g. ​ECW, ​MrSID.

Pseudo-Code