RFC 24: GDAL Progressive Data Support

Author: Norman Barker
Contact: nbarker@…
Status: Development

Summary

To provide an interface for data streaming support to GDAL by overloading the RasterIO function to include a callback function when there are buffer updates. The RFC focuses on JPIP but should be generic to apply to other streaming / progressive formats.

Definitions

JPIP: JPEG 2000 Interactive Protocol

Objective

To provide a callback function to allow users of a progressive format driver to receive notifications of updates to the underlying dataset for a particular requested region of the data. The notification mechanism should be accessible to all the swig wrappers.

Implementation

The implementation is a definition of an interface, in particular an overloaded function definition for RasterIO. Concrete implementations of this interface will follow. Currently the most convenient JPIP streaming developer library is ​Kakadu however since GDAL is also developer library, only stubs can be distributed to conform to Kakadu licensing (JP2KAK). Commercial vendors are also interested in using GDAL for streaming support and a standard interface will allow these commercial plugins to be incorporated. e.g. ​ECW, ​MrSID.

Discussion

​Tamas

1. It seems we are tending to spawn new threads in every RasterIO operations at driver level, which is quite inconvenient at the moment, therefore it should be considered with care. Will you allow RasterIO to be re-entered from the GDALProgFunc event handler or by another thread? Will you provide a copy of pBuff in GDALProgFunc or the same pointer will be passed back to the caller? How this buffer will be protected from the simultaneous access of the multiple threads?

Proposed Progressive Streaming Sequence of Events

Sequence shows GDALDataset, same sequence for GDALRasterBand