Changes between Version 58 and Version 59 of rfc24_progressive_data_support


Ignore:
Timestamp:
Apr 8, 2009, 4:37:35 PM (15 years ago)
Author:
normanb
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • rfc24_progressive_data_support

    v58 v59  
    1717To provide an interface to a streaming data source in a convenient manner for RasterIO operations within GDAL and to expose this interface via swig.
    1818
    19 Note this RFC originally explored using an Observer pattern as per implementations in Java [http://java.sun.com/j2se/1.4.2/docs/api/javax/imageio/event/IIOReadUpdateListener.html imageio update].  The RFC then also discussed the [http://www.gtk.org/api/2.6/glib/glib-Asynchronous-Queues.html Asynchronous Message Queue pattern], but after detailed discussions with [http://thread.gmane.org/gmane.comp.gis.gdal.devel/16409/focus=16630 Tamas Szekeres, Even Rouault, Adam Nowacki and Frank Warmerdam] an even simpler interface is proposed which wraps the more unpleasant developer tasks of operating with the streaming protocol but allows the user greater flexibility in interacting with the stream in their preferred language.  This is implemented by using a context object to act as an intermediary between the user layer and the format driver.  The approach recommended in the RFC is a half-way abstraction of the protocol as it requires some knowledge that this is a streaming format and how to optimise the driver from the user interface.  This is an acceptable, and actually desirable trade-off, since it allows the swig wrappers to implement high speed data access / display in a pattern suitable for the language.  The protocol may support a finer level of messaging than is available through this format driver.
     19== JPIPKAK - JPIP Streaming ==
    2020
    21 Note that we may wish to revisit the name the classes within this RFC since with this proposal the RasterIO, and block methods are blocking, hence there is not any  asynchronous communication in the format driver and this is delegated to the user code layer.
    22  
     21JPEG 2000 Interactive Protocol (JPIP) flexibility with respect to random access, code stream reordering and incremental decoding is highly exploitable in a networked environment allowing access to remote large files using limited bandwidth connections or high contention networks.
    2322
    24 == Implementation ==
     23=== JPIPKAK - JPIP Overview ===
    2524
    26 The implementation is a definition of an interface for streaming support data support within GDAL. Concrete implementations of this interface will follow.  Currently the most convenient JPIP streaming developer library is [http://www.kakadusoftware.com Kakadu] however since GDAL is also a developer library, only stubs can be distributed to conform to Kakadu licensing ([http://trac.osgeo.org/gdal/wiki/JP2KAK JP2KAK]).  Vendors are also interested in using GDAL for streaming support and a standard interface will allow these plugins to be incorporated. e.g. [http://www.gdal.org/frmt_jp2ecw.html ECW], [http://www.gdal.org/frmt_mrsid.html MrSID].
     25A brief overview of the JPIP event sequence is presented in this section, more information can be found at [http://www.jpeg.org/jpeg2000/j2kpart9.html JPEG 2000 Interactive Protocol (Part 9 – JPIP)] and the specification can (and should) be purchased from [http://www.iso.org/ ISO].
     26
     27An earlier version of JPEG 2000 Part 9 is available here [http://www.jpeg.org/public/fcd15444-9v2.pdf], noting the ISO copyright, diagrams are not replicated in this documentation.
     28
     29The JPIP protocol has been abstracted in this format driver, requests are made at the 1:1 resolution level.
    2730
    2831
    29 == Pseudo-Code ==
    30 
    31 '' interaction with the driver, the Definition below should be used for function names, not the pseudo-code''
    32 
    33 {{{
    34    GDALDataset *ds = GDALOpen(name, Access.GA_ReadOnly);
    35    
    36    // create an IO context for interaction with the remote server
    37    // the driver will initiate communication to the server and the context status to IO_PENDING
    38    // similar interaction with RasterBand
    39 
    40    GDALRasterIOContext *ctx = ds->CreateRasterIOContext(GF_Read, nDSXOff, nDSYOff, nDSXSize, nDSYSize, nBXSize, nBYSize, GDT_Byte, nBandCount, panBandMap, nPixelSpace, nLineSpace, nBandSpace);
    41 
    42    // optional : invoke a thread to execute communication to the server, and update the window when the thread terminates
    43    update = true;
    44 
    45    while (ctx-> status != IO_COMPLETE)
    46    {
    47       // Update the buffer with the next stage of data.  The driver is allowed to modify
    48       // the buffer within ReadNextBlock only
    49       // ReadNextBlock will wait for the next data response from the server
    50       ctx->ReadNextBlock(buffer, options);
    51 
    52       // check has the view updated, is the request cancelled
    53       if (bCancel || viewChanged)
    54       {
    55          update = false;
    56          ctx -> CancelIO();
    57          break;
    58       }
    59    }
    60 
    61    if (update)
    62      updateDisplay()
    63      
    64 }}}
    65 
    66 == Definition ==
    67 
    68 ''Header and dummy class file below''
    69 
    70 '''GDALAsyncDataset.h'''
    71 
    72 {{{
    73 /******************************************************************************
    74 *
    75 * Project: Progressive Renderer Client Driver
    76 * Purpose: Implementation of Dataset and RasterBand classes for JPIP and other
    77 *          progressive rendering protocols / formats
    78 * Author: Norman Barker, nbarker@ittvis.com
    79 *******************************************************************************
    80 */
    81 
    82 #include "gdal_pam.h"
    83 
    84 
    85 class GDALAsyncDataset;
    86 class GDALAsyncRasterBand;
    87 class GDALRasterIOContext;
    88 
    89 /**
    90 * status of the asynchronous stream
    91 */
    92 typedef enum
    93 {       
    94         IO_PENDING = 0,
    95         IO_UPDATE = 1,
    96         IO_ERROR = 2,
    97         IO_COMPLETE = 3
    98 } AsyncStatus;
    99 
    100 /**********************************************************************/
    101 /*                           GDALAsyncDataset                         */
    102 /**********************************************************************/
    103 class CPL_DLL GDALAsyncDataset : public GDALPamDataset
    104 {
    105         friend class GDALAsyncRasterBand;
    106         friend class GDALRasterIOContext;
    107 public:
    108         GDALAsyncDataset(void);
    109         virtual ~GDALAsyncDataset(){};
    110         virtual GDALRasterIOContext *CreateRasterIOContext(GDALRWFlag eRWFlag,
    111                                                 int nDSXOff, int nDSYOff,
    112                                                 int nDSXSize, int nDSYSize,
    113                                                 int nBXSize, int nBYSize,
    114                                                 GDALDataType eBDataType,
    115                                                 int nBandCount, int* panBandMap,
    116                                                 int nPixelSpace, int nLineSpace, int nBandSpace);
    117 
    118 
    119         CPLErr DeleteRasterIOContext(GDALRasterIOContext *);
    120 
    121         /*
    122         *  GDALAsyncDatasetOpen called with Access.GA_ReadOnly
    123         *  Implementations (with the exception of transactional jpip)
    124         *  are read only
    125         */
    126         static GDALDataset *GDALAsyncDatasetOpen(GDALOpenInfo *poOpenInfo);
    127 
    128 };
    129 
    130 /*********************************************************************/
    131 /*                        GDALAsyncRasterBand                        */
    132 /*********************************************************************/
    133 class CPL_DLL GDALAsyncRasterBand : public GDALPamRasterBand {
    134         friend class GDALAsyncDataset;
    135 public:
    136         GDALAsyncRasterBand(GDALAsyncDataset *parent_dataset, int band, double scale);
    137         virtual GDALRasterIOContext *CreateRasterIOContext(GDALRWFlag eRWFlag,
    138                                         int nDSXOff, int nDSYOff,
    139                                         int nDSXSize, int nDSYSize,
    140                                         int nBXSize, int nBYSize,
    141                                         GDALDataType eBDataType,
    142                                         int nPixelSpace, int nLineSpace);
    143         CPLErr DeleteRasterIOContext(GDALRasterIOContext *);
    144         virtual ~GDALAsyncRasterBand(){};
    145 };
    146 
    147 class CPL_DLL GDALRasterIOContext
    148 {
    149 private:
    150         AsyncStatus status;
    151 
    152 public:
    153         /**
    154         * Returns the current status of the asynchronous process
    155         * This value cannot be modified by the calling function, and
    156         * the value is true at the time of the function being called.
    157         */
    158         AsyncStatus GetStatus(){return status;}
    159 
    160         virtual CPLErr SetView(int xOff, int yOff, int xSize, int ySize, int fxSize, int fySize);
    161         virtual CPLErr RasterIO(void *pData, char **papszOptions, int nTimeoutMilliseconds = -1);
    162         virtual void CancelIO();
    163 };
    164 
    165 }}}
    166 
    167 '''GDALAsyncDataset.cpp'''
    168 {{{
    169 #include "GDALAsyncDataset.h"
    170 
    171 /************************************************************************/
    172 /* ==================================================================== */
    173 /*                             GDALAsyncDataset                         */
    174 /* ==================================================================== */
    175 /************************************************************************/
    176 
    177 /**
    178  * \class GDALAsyncDataset "GDALAsyncDataset.h"
    179  *
    180  */
    181 
    182 /************************************************************************/
    183 /*                            CreateRasterIOContext()                       */
    184 /************************************************************************/
    185 
    186 /**
    187 *  Create an IOContext for asynchronous operation.  The driver will initiate the transfer to the server
    188 * and set the current status for the user to read, @see AsyncStatus
    189 *
    190 * @param eRWFlag Read/Write flag, most format driver implementations will be Access.GA_ReadOnly.
    191 *                               JPIP transactional services will support GA_Update
    192 * @param nDSXOff request x offset into the server image at resolution 1:1,
    193                 top left corner is the image origin
    194 * @param nDSYOff request y offset into the server image at resolution 1:1,
    195                 top left corner is the image origin
    196 * @param nDSXSize request width of the server image at resolution 1:1
    197 * @param nDSYSize request height of the server image at resolution 1:1
    198 * @param nBXSize Requested frame width, JPIP will select the resolution level that best fits nBXSize, nBYSize
    199 * @param nBYSize Requested frame height, JPIP will select the resolution level that best fits nBXSize, nBYSize
    200 * @param eBDataType
    201 * @param nBandCount
    202 * @param panBandMap
    203 * @param nPixelSpace
    204 * @param nLineSpace
    205 * @param nBandSpace
    206 */
    207 GDALRasterIOContext *GDALAsyncDataset::CreateRasterIOContext(GDALRWFlag eRWFlag,
    208                                         int nDSXOff, int nDSYOff,
    209                                         int nDSXSize, int nDSYSize,
    210                                         int nBXSize, int nBYSize,
    211                                         GDALDataType eBDataType,
    212                                         int nBandCount, int* panBandMap,
    213                                         int nPixelSpace, int nLineSpace, int nBandSpace)
    214 {
    215         return 0;
    216 }
    217 
    218 /************************************************************************/
    219 /*                            DeleteRasterIOContext()                       */
    220 /************************************************************************/
    221 
    222 /**
    223 *  Clean up operations for child GDALRasterIOContext
    224 */
    225 CPLErr GDALAsyncDataset::DeleteRasterIOContext(GDALRasterIOContext *)
    226 {
    227         // return a warning, this method should be overridden
    228         return CE_Warning;
    229 }
    230 
    231 /************************************************************************/
    232 /* ==================================================================== */
    233 /*                             GDALAsyncRasterBand                         */
    234 /* ==================================================================== */
    235 /************************************************************************/
    236 
    237 /**
    238  * \class GDALAsyncRasterBand "GDALAsyncDataset.h"
    239  *
    240  */
    241 
    242 /************************************************************************/
    243 /*                            CreateRasterIOContext()                       */
    244 /************************************************************************/
    245 
    246 /**
    247 *  Create an IOContext for asynchronous operation.  The driver will initiate the transfer to the server
    248 * and set the current status for the user to read, @see AsyncStatus
    249 *
    250 * @param eRWFlag Read/Write flag, most format driver implementations will be Access.GA_ReadOnly.
    251 *                               JPIP transactional services will support GA_Update
    252 * @param nDSXOff request x offset into the server image at resolution 1:1,
    253                 top left corner is the image origin
    254 * @param nDSYOff request y offset into the server image at resolution 1:1,
    255                 top left corner is the image origin
    256 * @param nDSXSize request width of the server image at resolution 1:1
    257 * @param nDSYSize request height of the server image at resolution 1:1
    258 * @param nBXSize Requested frame width, JPIP will select the resolution level that best fits nBXSize, nBYSize
    259 * @param nBYSize Requested frame height, JPIP will select the resolution level that best fits nBXSize, nBYSize
    260 * @param eBDataType
    261 * @param nPixelSpace
    262 * @param nLineSpace
    263 */
    264 GDALRasterIOContext *GDALAsyncRasterBand::CreateRasterIOContext(GDALRWFlag eRWFlag,
    265                                         int nDSXOff, int nDSYOff,
    266                                         int nDSXSize, int nDSYSize,
    267                                         int nBXSize, int nBYSize,
    268                                         GDALDataType eBDataType,
    269                                         int nPixelSpace, int nLineSpace)
    270 {
    271         return 0;
    272 }
    273 
    274 /************************************************************************/
    275 /*                            DeleteRasterIOContext()                       */
    276 /************************************************************************/
    277 
    278 /**
    279 *  Clean up operations for child GDALRasterIOContext
    280 */
    281 CPLErr GDALAsyncRasterBand::DeleteRasterIOContext(GDALRasterIOContext *)
    282 {
    283     // return a warning, this method should be overridden
    284         return CE_Warning;
    285 }
    286 
    287 /************************************************************************/
    288 /* ==================================================================== */
    289 /*                             GDALRasterIOContext                      */
    290 /* ==================================================================== */
    291 /************************************************************************/
    292 
    293 /**
    294  * \class GDALRasterIOContext "GDALAsyncDataset.h"
    295  *
    296  */
    297 
    298 /**
    299 *  Set the currently request window into the dataset on the server at 1:1 resolution
    300 *
    301 * @param xOff request x offset into the server image at resolution 1:1,
    302                 top left corner is the image origin
    303 * @param yOff request y offset into the server image at resolution 1:1,
    304                 top left corner is the image origin
    305 * @param xSize request width of the server image at resolution 1:1
    306 * @param ySize request height of the server image at resolution 1:1
    307 * @param fxSize Requested frame width, JPIP will select the resolution level that best fits nBXSize, nBYSize
    308 * @param fySize Requested frame height, JPIP will select the resolution level that best fits nBXSize, nBYSize
    309 */
    310 CPLErr GDALRasterIOContext::SetView(int xOff, int yOff, int xSize, int ySize, int fxSize, int fySize)
    311 {
    312         // return a warning, this method should be overridden
    313         return CE_Warning;
    314 }
    315 
    316 /*************************************************************************/
    317 /* ===================================================================== */
    318 /*                       CancelIO()                                      */
    319 /* ===================================================================== */
    320 /*************************************************************************/
    321 
    322 /**
    323 *  Cancel the communication channel to the remote server
    324 *
    325 */
    326 void GDALRasterIOContext::CancelIO()
    327 {
    328 }
    329 
    330 /*******************************************************************************/
    331 /* =========================================================================== */
    332 /*                     RasterIO()                                              */
    333 /* =========================================================================== */
    334 /*******************************************************************************/
    335 
    336 /**
    337 * Read a view from the remote image on the server
    338 * @param pData buffer to fill
    339 * @param nTimeoutMilliseconds connection timeout
    340 * @papszOptions options for the driver implementation e.g. Quality="Xxx" to specify maximum jpip quality
    341 */
    342 CPLErr RasterIO(void *pData, char **papszOptions, int nTimeoutMilliseconds = -1)
    343 {
    344         // return a warning, this method should be overridden
    345         return CE_Warning;
    346 }
    347 
    348 }}}