Changes between Version 8 and Version 9 of SoCWMSDriver


Ignore:
Timestamp:
Jul 16, 2007, 8:57:02 AM (17 years ago)
Author:
nowakpl
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • SoCWMSDriver

    v8 v9  
    33== OGC WMS Driver ==
    44
    5 Primary goal of the project is to allow remote access to geospatial image data using WMS protocol. The driver will be compatible at least with version 1.3.0, 1.1.1 and 1.1.0 specifications.
    6 Secondary goal is to allow remote access using other WMS-like or tile protocols including WMS-C, TMS, TileService.
    7 
    8 === Details ===
    9  * server address and parameters via xml file and embedded in dataset name
    10   * complete information in xml file
    11    * example1.xml
    12 {{{
    13 <GDAL_WMS_Driver>
    14   <Service name="WMS">
    15     <Version>1.3.0</Version>
    16     <ServerUrl>http://127.0.0.1/mapserv.cgi</ServerUrl>
    17     <CRS>CRS:83</CRS>
    18     <ImageFormat>image/png</ImageFormat>
    19     <Layers>layer1,layer2</Layers>
    20     <Styles>style1,style2</Styles>
    21     <DataWindow>
    22       <MinX>-180.0</MinX>
    23       <MaxX>180.0</MaxX>
    24       <MinY>-90.0</MinY>
    25       <MaxY>90.0</MaxY>
    26       <ResolutionX>0.01</ResolutionX>
    27       <ResolutionY>0.01</ResolutionY>
    28     </DataWindow>
    29     <DataAccess type="tiled">
    30       <OriginX>-180.0</OriginX>
    31       <OriginY>-90.0</OriginY>
    32       <TileSizeX>256</TileSizeX>
    33       <TileSizeY>256</TileSizeY>
    34     </DataAccess>
    35   </Service>
    36   <Cache>
    37     <Path>/tmp/example1_cache</Path>
    38   </Cache>
    39 </GDAL_WMS_Driver>
    40 }}}
    41    * dataset name
    42 {{{
    43 WMS:example1.xml
    44 }}}
    45   * minimal information in xml file (other values extracted from GetCapabilities response / defaults)
    46    * example2.xml
    47 {{{
    48 <GDAL_WMS_Driver>
    49   <Service name="WMS">
    50     <ServerUrl>http://127.0.0.1/mapserv.cgi</ServerUrl>
    51     <Layers>layer1,layer2</Layers>
    52   </Service>
    53 </GDAL_WMS_Driver>
    54 }}}
    55    * dataset name
    56 {{{
    57 WMS:example2.xml
    58 }}}
    59   * minimal information in xml file, layers selection in dataset name
    60    * example3.xml
    61 {{{
    62 <GDAL_WMS_Driver>
    63   <Service name="WMS">
    64     <ServerUrl>http://127.0.0.1/mapserv.cgi</ServerUrl>
    65   </Service>
    66 </GDAL_WMS_Driver>
    67 }}}
    68    * dataset name
    69 {{{
    70 WMS:example3.xml:Service,Layers=layer1,layer2,layer3
    71 }}}
    72 
    73  * issue and parse GetCapabilities requests
    74   * provide simplified human-readable version for gdalinfo
    75   * store layer/style name, description, legend url, etc. in GDAL dataset meta-data fields
    76 
    77  * two image data access methods
    78   * tiled: data is requested in small blocks, cached
    79   * immediate: one request per RasterIO call, not cached
    80 
    81 === External links ===
    82  * [http://www.opengeospatial.org/standards/wms#downloads OGC WMS Specification]
    83  * [http://wiki.osgeo.org/index.php/Tile_Map_Service_Specification TMS Specification]
    84  * [http://wiki.osgeo.org/index.php/WMS_Tiling_Client_Recommendation WMS Tiling Client Recommendation]
     5The driver is composed of interface layer between GDAL and mini-drivers, and many small mini-drivers. Mini-driver only task is to provide image locations, actual downloads and returning data is handled by interface layer. Example:
     6 1. User requests (via RasterIO call) 400x400 image block starting at pixel (500, 700).
     7 2. Mini-driver is asked to provide list of images to download and returns:
     8  1. upper-left corner: (500, 500), lower-right corner: (1000, 1000)
     9url: http://example.com/?width=500&height=500&bbox=5,5,10,10
     10  2. upper-left corner: (500, 1000), lower-right corner: (1000, 1500)
     11url: http://example.com/?width=500&height=500&bbox=5,10,10,15
     12 3. The images are downloaded, resampled, mosaicked, cropped to fit the user request.
     13The interface layer implements:
     14{{{
     15        class GDALWMSDataset : public GDALPamDataset {
     16            virtual CPLErr IRasterIO( ... );
     17            virtual CPLErr AdviseRead( ... );
     18            virtual const char *GetProjectionRef();
     19            virtual CPLErr GetGeoTransform( ... );
     20        };
     21
     22        class GDALWMSRasterBand : public GDALPamRasterBand {
     23            virtual CPLErr IReadBlock( ... );
     24            virtual CPLErr IRasterIO( ... );
     25            virtual CPLErr AdviseRead( ... );
     26            virtual int HasArbitraryOverviews();
     27            virtual int GetOverviewCount();
     28            virtual GDALRasterBand *GetOverview( ... );
     29        };
     30}}}
     31HasArbitraryOverviews always returns true. Overview count is set in configuration file, each overview layer has half the resolution of previous layer. GetProjectionRef returns projection as specified in configuration file. GetGeoTransform returns geo-transform constructed from data window. IRasterIO, AdviseRead and IReadBlock are translated into common request format:
     32{{{
     33        class ImageRequest {
     34        public:
     35            double m_x0, m_y0; // geo-referenced upper-left corner
     36            double m_x1, m_y1; // geo-referenced lower-right corner
     37            int m_xsize, m_ysize; // width and height in pixels
     38        };
     39
     40        class ImageBufferInfo { // see GDALDataset::RasterIO documentation
     41        public:
     42            void *m_data;
     43            int m_xsize, m_ysize;
     44            GDALDataType m_data_type;
     45            int m_band_count;
     46            int *m_band_map;
     47            int m_pixel_space;
     48            int m_line_space;
     49            int m_band_space;
     50        };
     51}}}
     52Dataset open
     53 1. open, verify and read XML configuration file into CPL MiniXML tree
     54 2. update XML tree with arguments specified in dataset name
     55 3. create instance of mini-driver and pass in the XML tree
     56Dataset image request
     57 1. validate the request
     58 2. check if request can be satisfied from block cache only and return the data to user if possible
     59 3. call into mini-driver getting back list of image locations
     60 4. download the images, if allowed try in cache first
     61 5. update block cache wherever possible (resolution match, complete coverage)
     62 6. return the data to user
     63Download cache
     64If enabled in XML configuration file and allowed by mini-driver, downloaded images will be written to disk for future use. This should reduce load on the WMS server or even allow off-line operation.
     65HTTP download queue
     66Download queue is implemented using libcurl (http://curl.haxx.se/libcurl/), running in dedicated background thread to allow data preloading (AdviseRead or guessed). Can be configured to use HTTP pipelining and multiple connections.
     67XML configuration file
     68{{{
     69        <GDALWMS>
     70          <HTTP> ... </HTTP>
     71          <DataSource> ... </DataSource>
     72          <DataWindow> ... </DataWindow>
     73          <Cache> ... </Cache>
     74        </GDALWMS>
     75}}}
     76<HTTP> configures the HTTP download queue.
     77{{{
     78        <HTTP>
     79          <MaxPipeline> ... </MaxPipeline>    number of requests on single connection     <MaxParallel> ... </MaxParallel>    number of parallel connections
     80          <UserAgent> ... </UserAgent>        custom user agent header
     81          <User> ... </User>                  http authentication, user
     82          <Password> ... </Password>          http authentication, password
     83          <Proxy> ... </Proxy>                proxy support, see curl for docs
     84          <ProxyPort> ... </ProxyPort>       
     85          <ProxyType> ... </ProxyType>       
     86          <ProxyUser> ... </ProxyUser>       
     87          <ProxyPassword> ... </ProxyPassword>
     88        </HTTP>
     89}}}
     90<DataSource> specifies name and configuration for mini-driver.
     91{{{
     92        <DataSource driver=”some mini-driver”>
     93          ... mini-driver specific configuration ...
     94        </DataSource>
     95}}}
     96<DataWindow> specifies projection, extents and resolution of data as visible by GDAL. Image size is in pixels, upper-left and lower-right corners are in specified projection. Overview count is optional and can be set to 0, default is calculated so last overview is bigger than 512x512. Block size is optional and defaults to 512x512.
     97{{{
     98        <DataWindow>
     99          <Projection> ... </Projection>
     100          <UpperLeftX> ... </UpperLeftX>
     101          <UpperLeftY> ... </UpperLeftY>
     102          <LowerRightX> ... </LowerRightX>
     103          <LowerRightY> ... </LowerRightY>
     104          <SizeX> ... </SizeX>
     105          <SizeY> ... </SizeY>
     106          <OverviewCount> ... </OverviewCount>
     107          <BlockSizeX> ... </BlockSizeX>
     108          <BlockSizeY> ... </BlockSizeY>
     109        </DataWindow>
     110}}}
     111Example data window (red) in global WMS layer (blue), geographic projection.
     112{{{
     113        <DataWindow>
     114          <Projection>EPSG:4326</Projection>
     115          <UpperLeftX>-120.0</UpperLeftX>
     116          <UpperLeftY>60.0</UpperLeftY>
     117          <LowerRightX>40.0</LowerRightX>
     118          <LowerRightY>-40.0</LowerRightY>
     119          <SizeX>4000</SizeX>
     120          <SizeY>2500</SizeY>
     121          <BlockSizeX>500</BlockSizeX>
     122          <BlockSizeY>500</BlockSizeY>
     123        </DataWindow>
     124}}}
     125<Cache> specifies location of disk cache. If location is not set cache is disabled.
     126{{{
     127        <Cache>
     128          <Path>/some/directory</Path>
     129        </Cache>
     130}}}
     131Mini-driver
     132Image download request, filled in by mini-driver and returned to interface layer.
     133{{{
     134        class ImageDownloadRequest {
     135        public:
     136            CPLString m_url;       // image url to download
     137            double m_x0, m_y0;     // georeferenced upper-left
     138            double m_x1, m_y1;     // georeferenced bottom-right
     139            int m_xsize, m_ysize;  // width and height in pixels
     140            bool m_use_cache;      // should the image be cached / read from cache
     141        };
     142
     143        typedef std::vector<ImageDownloadRequest> ImageDownloadRequestList;
     144}}}
     145Mini-driver
     146{{{
     147        class MiniDriver {
     148        public:
     149            MiniDriver(CPLXMLNode *config);
     150            virtual ~MiniDriver();
     151
     152        public:
     153            // Process image request and add image download requests to the list
     154            virtual void AppendImageDownloadRequestList(
     155                ImageDownloadRequestList &out, const ImageRequest &in);
     156        };
     157}}}
     158Mini-driver factory, used to create instance of given mini-driver.
     159{{{
     160        class MiniDriverFactory {
     161        public:
     162            MiniDriverFactory();
     163            virtual ~MiniDriverFactory();
     164
     165        public:
     166            // create mini-driver instance, basically:
     167            // return new ExampleMiniDriver(config);
     168            virtual MiniDriver* New(CPLXMLNode *config);
     169
     170            // destroy mini-driver instance, basically:
     171            // delete instance;
     172            virtual void Delete(MiniDriver *instance);
     173
     174        public:
     175            const std::string &GetName() {
     176                return m_name;
     177            }
     178
     179        protected:
     180            // <DataSource name=”...”> in configuration file
     181            std::string m_name;
     182        };
     183}}}
     184
     185Planned mini-drivers
     186 * OGC WMS (http://www.opengeospatial.org/standards/wms)
     187 * TMS (http://wiki.osgeo.org/index.php/Tile_Map_Service_Specification)
     188 * OnEarth Tiled WMS (http://onearth.jpl.nasa.gov/tiled.html)
     189 * WMS-C (http://tilecache.org/, http://wiki.osgeo.org/index.php/WMS_Tiling_Client_Recommendation)
     190 * World Wind tiled datasets, both remote http and local file versions (http://issues.worldwind.arc.nasa.gov/confluence/download/attachments/394/world+wind+tile+systemt.gif)
     191 * TileService (no docs, similar to World Wind, example http://s0.tileservice.worldwindcentral.com/getTile?interface=map&version=1&dataset=bmng.topo.bathy.200401&level=0&x=0&y=0)
     192OGC WMS mini-driver
     193The driver will not issue GetCapabilities requests, all information has to be stored in XML configuration file. Small utility program will be provided (probably a script) to handle GetCapabilities requests and write configuration file.
     194Arbitrary overviews handled by WMS server.
     195Custom arguments in server URL (for example WMS Time)
     196Many download patterns:
     197 1. one download (light green) per image request (dark green), partial block cache (gray), disk cache not allowed
     198 2. one download per image request, extended to block cache grid, disk cache not allowed
     199 3. many downloads per image request (tiled), full block cache, disk cache allowed
     200OGC WMS mini-driver configuration
     201{{{
     202        <DataSource driver=”WMS”>
     203          <Version>1.3.0</Version>
     204          <ServerUrl>http://127.0.0.1/mapserv.cgi</ServerUrl>
     205          <CRS>CRS:83</CRS>
     206          <ImageFormat>image/png</ImageFormat>
     207          <Layers>layer1,layer2</Layers>
     208          <Styles>style1,style2</Styles>
     209          <CustomArgs>
     210            <time>2000-01-01T</time>
     211          </CustomArgs>
     212        </DataSource>
     213}}}
     214Example mini-driver implementation
     215Mini-driver for a WMS-like service with requests in format:
     216http://server/file?width=...&height=...&bbox=...
     217{{{
     218        MyMiniDriver::MyMiniDriver(CPLXMLNode *config) {
     219            m_server = CPLGetXMLValue(config, "ServerURL", "");
     220        }
     221
     222        MyMiniDriver::~MyMiniDriver() {
     223        }
     224
     225        void MyMiniDriver::AppendImageDownloadRequestList(
     226                ImageDownloadRequestList &out, const ImageRequest &in) {
     227            ImageDownloadRequest req;
     228            req.m_use_cache = false;
     229            req.m_x0 = in.m_x0;
     230            req.m_y0 = in.m_y0;
     231            req.m_x1 = in.m_x1;
     232            req.m_y1 = in.m_y1;
     233            req.m_xsize = in.m_xsize;
     234            req.m_ysize = in.m_ysize;
     235            req.m_url.Printf("%s?width=%d&height=%d&bbox=%g,%g,%g,%g",
     236                    m_server.c_str(), in.m_xsize, in.m_ysize,
     237                    in.m_x0, in.m_y0, in.m_x1, in.m_y1);
     238            out.push_back(req);
     239        }
     240
     241        MyMiniDriverFactory::MyMiniDriverFactory() {
     242            m_name = "MyWMS";
     243        }
     244
     245        MyMiniDriverFactory::~MiniDriverFactory() {
     246        }
     247
     248        MiniDriver* MyMiniDriverFactory::New(CPLXMLNode *p) {
     249            return new MyMiniDriver(p);
     250        }
     251
     252        void MyMiniDriverFactory::Delete(MiniDriver *p) {
     253            delete p;
     254        }
     255}}}
     256Example XML configuration file for MyWMS mini-driver:
     257{{{
     258        <GDALWMS>
     259          <DataSource name=”MyWMS”>
     260            <ServerURL>http://localhost/mywms</ServerURL>
     261          </DataSource>
     262          ...
     263        </GDALWMS>
     264}}}
    85265
    86266=== People ===
    … …  
    94274
    95275 * What protocol versions will be supported.  I think WMS 1.1.0 is the most important but 1.0.0, and 1.3.0 are also desirable (FrankW)
     276  * As far as GetMap requests are concerned all three are almost equal (Nowak)