Changes between Initial Version and Version 1 of rfc24_jpipkak

Feb 2, 2010, 12:07:01 PM (14 years ago)



  • rfc24_jpipkak

    v1 v1  
     1= JPIPKAK: Background on JPIP implementation of RFC 24 =
     3== JPIPKAK - JPIP Streaming ==
     5JPEG 2000 Interactive Protocol (JPIP) flexibility with respect to random access, code stream reordering and incremental decoding is highly exploitable in a networked environment allowing access to remote large files using limited bandwidth connections or high contention networks.
     7=== JPIPKAK - JPIP Overview ===
     9A brief overview of the JPIP event sequence is presented in this section, more information can be found at [ JPEG 2000 Interactive Protocol (Part 9 – JPIP)] and the specification can (and should) be purchased from [ ISO].
     11An earlier version of JPEG 2000 Part 9 is available here [], noting the ISO copyright, diagrams are not replicated in this documentation.
     13The JPIP protocol has been abstracted in this format driver, requests are made at the 1:1 resolution level.
     17 1. Initial JPIP request for a target image, a target id, a session over http, data to be returned as a jpp-stream are requested and a maximum length is put on the response. In this case no initial window is requested, though it can be. Server responds with a target identifier that can be used to identify the image on the server and a JPIP-cnew response header which includes the path to the JPIP server which will handle all future requests and a cid session identifier. A session is required so that that the server can model the state of the client connection, only sending the data that is required.
     18 1. Client requests particular view windows on the target image with a maximum response length and includes the session identifier established in the previous communication. 'fsiz' is used to identify the resolution associated with the requested view-window. The values 'fx' and 'fy' specify the dimensions of the desired image resolution. 'roff' is used to identify the upper left hand corner off the spatial region associated with the requested view-windw. 'rsiz' is used to identify the horizontal and vertical extents of the spatial region associated with the requested view-window.
     20=== JPIPKAK - approach ===
     22The JPIPKAK driver uses an approach that was first demonstrated here, [ J2KViewer], by Juan Pablo Garcia Ortiz of separating the communication layer (socket / http) from the Kakadu kdu_cache object. Separating the communication layer from the data object is desirable since it allows the use of optimized http client libraries such as libcurl, Apache HttpClient (note that jportiz used a plain Java socket) and allows SSL communication between the client and server.
     24Kakadu's implementation of client communication with a JPIP server uses a socket, and this socket connection holds the state for this client session. A client session with Kakadu can be recreated using the JPIP cache operations between client and server, but no use of traditional HTTP cookies is supported since JPIP is neutral to the transport layer.
     26The JPIPKAK driver is written using a HTTP client library with the Kakadu cache object and supports optimized communication with a JPIP server (which may or may not support HTTP sessions) and the high performance of the kakadu kdu_region_decompressor.
     30=== JPIPKAK - implementation ===
     32 The implementation supports the GDAL C++ and C API, and provides an initial SWIG wrapper for this driver with a Java ImageIO and Python example using SWIG.
     36The driver uses a simple threading model to support requesting reads of the data and remote fetching. This threading model supports two separate client windows, with just one connection to the server. Requests to the server are multiplexed to utilize available bandwidth efficiently. The client identifies these windows by using “0” (low) or “1” (high) values to a “PRIORITY” metadata request option.
     38''Note: SSL support''
     40''If the client is built with support for SSL, then driver determines whether to use SSL if the request is a jpips:// protocol as opposed to jpip://.''
     41''Note that the driver does not verify server certificates using the Curl certificate bundle and is currently set to accept all SSL server certificates.''
     43''Note: libCurl''
     45''JPIP sets client/server values using HTTP headers, modifications have been made to the GDAL HTTP portability library to support this.''
     491.  GDALGetDatasetDriver
     51Fetch the driver to which this dataset relates.
     532. Open
     55If the filename contained in the `GDALOpenInfo` object has a case insensitive URI scheme of jpip or jpips the `JPIPKAKDataset` is created and initialised, otherwise NULL is returned.
     573. Initialize
     59Initialisation involves making an initial connection to the JPIP Server to establish a session and to retrieve the initial metadata about the image (ref. JPIP Sequence Diagram).
     61If the connection fails, the function returns false and the Open function returns NULL indicating that opening the dataset with this driver failed.
     63If the connection is successful, then subsequent requests to the JPIP server are made to retrieve all the available metadata about the image. Metadata items are set using the `GDALMajorObject->SetMetadataItem` in the "JPIP" domain.
     65If the metadata returned from the server includes GeoJP2 UUID box, or a GMLJP2 XML box then this metadata is parsed and sets the geographic metadata of this dataset.
     674. GDALGetMetadata
     69C API to `JPIPKAKDataset->GetMetadata`
     715. !GetMetadata
     73returns metadata for the "JPIP" domain, keys are "JPIP_NQUALITYLAYERS", "JPIP_NRESOLUTIONLEVELS", "JPIP_NCOMPS" and "JPIP_SPRECISION"
     756. GDALEndAsyncRasterIO
     77If the asynchronous raster IO is active and not required, the C API calls `JPIPKAKDataset->EndAsyncRasterIO`
     797. EndAsyncRasterIO
     81The JPIPKAKAsyncRasterIO object is deleted
     838. delete
     859. GDALBeginAsyncRasterIO
     87C API to `JPIPKAKDataset->BeginAsyncRasterIO`
     8910. BeginAsyncRasterIO
     91The client has set the requested view window at 1:1 and have optionally set the discard level, quality layers and thread priority metadata items.
     9311. Create
     95Creates a JPIPKAKAsyncRasterIO Object
     9712. Start
     99Configures the kakadu machinery and starts a background thread (if not already running) to communicate to the server the current view window request. The background thread results in the kdu_cache object being updated until the JPIP server sends an "End Of Response" (EOR) message for the current view window request.
     10113. GDALLockBuffer
     103C API to !LockBuffer
     10514. !LockBuffer
     107Not implemented in `JPIPKAKAsyncRasterIO`, a lock is acquired in `JPIPKAKAsyncRasterIO->GetNextUpdatedRegion`
     10915. GDALGetNextUpdatedRegion
     111C API to !GetNextUpdatedRegion
     11316. !GetNextUpdatedRegion
     115The function decompresses the available data to generate an image (according to the dataset buffer type set in `JPIPKAKDataset->BeginAsyncRasterIO`) The window width, height (at the requested discard level) decompressed is returned in the region pointer and can be rendered by the client. The status of the rendering operation is one of `GARIO_PENDING`, `GARIO_UPDATE`, `GARIO_ERROR`, `GARIO_COMPLETE` from the `GDALAsyncStatusType` structure. `GARIO_UPDATE`, `GARIO_PENDING` require more reads of `GetNextUpdatedRegion` to get the full image data, this is the progressive rendering of JPIP. `GARIO_COMPLETE` indicates the window is complete.
     117`GDALAsyncStatusType` is a structure used by `GetNextUpdatedRegion` to indicate whether the function should be called again when either kakadu has more data in its cache to decompress, or the server has not sent an End Of Response (EOR) message to indicate the request window is complete.
     119The region passed into this function is passed by reference, and the caller can read this region when the result returns to find the region that has been decompressed. The image data is packed into the buffer, e.g. RGB if the region requested has 3 components.
     12117. GDALUnlockBuffer
     123C API to !UnlockBuffer
     12518. !UnlockBuffer
     127Not implemented in `JPIPKAKAsyncRasterIO`, a lock is acquired in `JPIPKAKAsyncRasterIO->GetNextUpdatedRegion`
     12919. Draw
     131Client renders image data
     13320. GDALLockBuffer
     13521. !LockBuffer
     13722. GDALGetNextUpdatedRegion
     13923. !GetNextUpdatedRegion
     14124. GDALUnlockBuffer
     14325. !UnlockBuffer
     14526. Draw
     147=== JPIPKAK - installation requirements ===
     149 * [ Libcurl 7.9.4]
     150 * [ OpenSSL 0.9.8K](if SSL is required, a JPIPS connection)
     151 * [ Kakadu] (tested with v5.2.6 and v6)
     153Currently only a Windows makefile is provided, however this should compile on Linux as well as there are no Windows dependencies.
     155See Also:
     157 * [ JPEG 2000 Interactive Protocol (Part 9 – JPIP)]
     158 * []
     159 * [ Kakadu Software ]
     160 * [ IAS demo (example JPIP(S) streams)]