Changes between Version 4 and Version 5 of Future/TileServiceEnhancements


Ignore:
Timestamp:
Jun 26, 2013, 6:16:33 AM (11 years ago)
Author:
jng
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • Future/TileServiceEnhancements

    v4 v5  
    134134MgHttpRequestMetadata and MgHttpHeader classes are not currently used in any of the existing CGI/Apache/ISAPI handlers. We should use them for this purpose.
    135135
     136= Re-usable tile sets =
     137
     138Currently the concept of a tile set is tightly-bound to the Map Definition that severely limits its re-usability between other maps.
     139
     140The second problem is that because a tile set is bound to a Map Definition, we have the all too common problem of shotgun tile cache invalidation on even the most minute changes in the Map Definition and/or upstream dependent resources.
     141
     142== The TileSetDefinition resource ==
     143
     144Ideally the tile set should be modelled not as a part of a Map Definition, but as a separate resource.
     145
     146A tentative schema for this resource would look something like this:
     147
     148{{{
     149<?xml version="1.0" encoding="UTF-8"?>
     150<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" attributeFormDefault="unqualified">
     151  <xs:element name="TileSetDefinition" minOccurs="0">
     152    <xs:annotation>
     153      <xs:documentation>Defines a tile cache</xs:documentation>
     154    </xs:annotation>
     155    <xs:complexType>
     156      <xs:sequence>
     157        <xs:element name="TileStoreParameters" type="TileStoreParametersType">
     158           <xs:annotation>
     159            <xs:documentation>Defines the parameters to access the tile cache</xs:documentation>
     160          </xs:annotation>
     161        </xs:element>
     162        <xs:element name="Extents" type="Box2DType">
     163          <xs:annotation>
     164            <xs:documentation>A bounding box around the area of the tile cache</xs:documentation>
     165          </xs:annotation>
     166        </xs:element>
     167        <xs:element name="CoordinateSystem" type="xs:string">
     168          <xs:annotation>
     169            <xs:documentation>The coordinate system as WKT used by the TileSetDefinition</xs:documentation>
     170          </xs:annotation>
     171        </xs:element>
     172        <xs:element name="FiniteDisplayScale" type="xs:double" maxOccurs="unbounded">
     173          <xs:annotation>
     174            <xs:documentation>The display scales that the base map layers will have tiles available. Applies to the HTML viewer.</xs:documentation>
     175          </xs:annotation>
     176        </xs:element>
     177        <xs:element name="BaseMapLayerGroup" type="BaseMapLayerGroupCommonType" minOccurs="1" maxOccurs="unbounded">
     178          <xs:annotation>
     179            <xs:documentation>A group of layers that is used to compose a tiled layer in the HTML viewer</xs:documentation>
     180          </xs:annotation>
     181        </xs:element>
     182      </xs:sequence>
     183    </xs:complexType>
     184  </xs:element>
     185  <xs:complexType name="TileStoreParametersType">
     186    <xs:annotation>
     187      <xs:documentation>TileStoreParameters defines the parameters of this tile cache.</xs:documentation>
     188    </xs:annotation>
     189    <xs:sequence>
     190      <xs:element name="TileWidth" type="xs:integer">
     191        <xs:annotation>
     192          <xs:documentation>The width of tile images in this tile cache</xs:documentation>
     193        </xs:annotation>
     194      </xs:element>
     195      <xs:element name="TileHeight" type="xs:integer">
     196        <xs:annotation>
     197          <xs:documentation>The height of tile images in this tile cache</xs:documentation>
     198        </xs:annotation>
     199      </xs:element>
     200      <xs:element name="ImageFormat" type="xs:string">
     201        <xs:annotation>
     202          <xs:documentation>The image format of tile images in this tile cache</xs:documentation>
     203        </xs:annotation>
     204      </xs:element>
     205      <xs:element name="TileProvider" type="xs:string">
     206        <xs:annotation>
     207          <xs:documentation>The tile image provider</xs:documentation>
     208        </xs:annotation>
     209      </xs:element>
     210      <xs:element name="Parameter" type="NameValuePairType" minOccurs="0" maxOccurs="unbounded">
     211        <xs:annotation>
     212          <xs:documentation>Collection of name value pairs for connecting to the tile image provider</xs:documentation>
     213        </xs:annotation>
     214      </xs:element>
     215    </xs:sequence>
     216  </xs:complexType>
     217  <xs:complexType name="NameValuePairType">
     218    <xs:annotation>
     219      <xs:documentation>A type describing name and value pairs</xs:documentation>
     220    </xs:annotation>
     221    <xs:sequence>
     222      <xs:element name="Name" type="xs:string">
     223        <xs:annotation>
     224          <xs:documentation>Text for the name of parameter</xs:documentation>
     225        </xs:annotation>
     226      </xs:element>
     227      <xs:element name="Value" type="xs:string">
     228        <xs:annotation>
     229          <xs:documentation>Text for value of parameter</xs:documentation>
     230        </xs:annotation>
     231      </xs:element>
     232      <xs:element name="ExtendedData1" type="ExtendedDataType" minOccurs="0"/>
     233    </xs:sequence>
     234  </xs:complexType>
     235  <xs:complexType name="Box2DType">
     236    <xs:annotation>
     237      <xs:documentation>Box2D encapsulates the the coordinates of a box in 2-D space</xs:documentation>
     238    </xs:annotation>
     239    <xs:sequence>
     240      <xs:element name="MinX" type="xs:double">
     241        <xs:annotation>
     242          <xs:documentation>Minimum x-coordinate</xs:documentation>
     243        </xs:annotation>
     244      </xs:element>
     245      <xs:element name="MaxX" type="xs:double">
     246        <xs:annotation>
     247          <xs:documentation>Maximum x-coordinate</xs:documentation>
     248        </xs:annotation>
     249      </xs:element>
     250      <xs:element name="MinY" type="xs:double">
     251        <xs:annotation>
     252          <xs:documentation>Minimum y-coordinate</xs:documentation>
     253        </xs:annotation>
     254      </xs:element>
     255      <xs:element name="MaxY" type="xs:double">
     256        <xs:annotation>
     257          <xs:documentation>Maximum y-coordinate</xs:documentation>
     258        </xs:annotation>
     259      </xs:element>
     260    </xs:sequence>
     261  </xs:complexType>
     262  <xs:complexType name="BaseMapLayerType">
     263    <xs:annotation>
     264      <xs:documentation>BaseMapLayerType encapsulates the properties of a BaseMapLayer.</xs:documentation>
     265    </xs:annotation>
     266    <xs:sequence>
     267      <xs:element name="Name" type="xs:string">
     268        <xs:annotation>
     269          <xs:documentation>Name of the MapLayer</xs:documentation>
     270        </xs:annotation>
     271      </xs:element>
     272      <xs:element name="ResourceId" type="xs:string">
     273        <xs:annotation>
     274          <xs:documentation>ResourceId of the MapLayer</xs:documentation>
     275        </xs:annotation>
     276      </xs:element>
     277      <xs:element name="Selectable" type="xs:boolean">
     278        <xs:annotation>
     279          <xs:documentation>Whether or not the Layer can be selected</xs:documentation>
     280        </xs:annotation>
     281      </xs:element>
     282      <xs:element name="ShowInLegend" type="xs:boolean">
     283        <xs:annotation>
     284          <xs:documentation>Whether or not the Layer should be shown in the legend</xs:documentation>
     285        </xs:annotation>
     286      </xs:element>
     287      <xs:element name="LegendLabel" type="xs:string">
     288        <xs:annotation>
     289          <xs:documentation>Label to be shown for the Layer in the legend</xs:documentation>
     290        </xs:annotation>
     291      </xs:element>
     292      <xs:element name="ExpandInLegend" type="xs:boolean">
     293        <xs:annotation>
     294          <xs:documentation>Whether or not the Layer should be expanded in the legend.</xs:documentation>
     295        </xs:annotation>
     296      </xs:element>
     297    </xs:sequence>
     298  </xs:complexType>
     299  <xs:complexType name="MapLayerGroupCommonType">
     300    <xs:annotation>
     301      <xs:documentation>MapLayerGroupCommonType is a common subclass of MapLayerGroupCommonType and BaseMapLayerGroupCommonType</xs:documentation>
     302    </xs:annotation>
     303    <xs:sequence>
     304      <xs:element name="Name" type="xs:string">
     305        <xs:annotation>
     306          <xs:documentation>The name of this LayerGroup</xs:documentation>
     307        </xs:annotation>
     308      </xs:element>
     309      <xs:element name="Visible" type="xs:boolean">
     310        <xs:annotation>
     311          <xs:documentation>Whether this group's visiblity should be visible or not when it first comes into range</xs:documentation>
     312        </xs:annotation>
     313      </xs:element>
     314      <xs:element name="ShowInLegend" type="xs:boolean">
     315        <xs:annotation>
     316          <xs:documentation>Whether or not the LayerGroup should be shown in the legend</xs:documentation>
     317        </xs:annotation>
     318      </xs:element>
     319      <xs:element name="ExpandInLegend" type="xs:boolean">
     320        <xs:annotation>
     321          <xs:documentation>Whether or not the LayerGroup should be initially expanded in the legend</xs:documentation>
     322        </xs:annotation>
     323      </xs:element>
     324      <xs:element name="LegendLabel" type="xs:string">
     325        <xs:annotation>
     326          <xs:documentation>Label to be shown for the LayerGroup in the legend</xs:documentation>
     327        </xs:annotation>
     328      </xs:element>
     329    </xs:sequence>
     330  </xs:complexType>
     331  <xs:complexType name="BaseMapLayerGroupCommonType">
     332    <xs:annotation>
     333      <xs:documentation>BaseMapLayerGroupCommonType encapsulates the properties of a BaseMapLayerGroup. It extends MapLayerGroupCommonType by holding the layers in the group.  The base map layer groups defines what layers are used to render a tile set in the HTML viewer.</xs:documentation>
     334    </xs:annotation>
     335    <xs:complexContent>
     336      <xs:extension base="MapLayerGroupCommonType">
     337        <xs:sequence>
     338          <xs:element name="BaseMapLayer" type="BaseMapLayerType" minOccurs="0" maxOccurs="unbounded">
     339            <xs:annotation>
     340              <xs:documentation>The layers that are part of this group. The order of the layers represents the draw order, layers first is the list are drawn over top of layers later in the list.</xs:documentation>
     341            </xs:annotation>
     342          </xs:element>
     343        </xs:sequence>
     344      </xs:extension>
     345    </xs:complexContent>
     346  </xs:complexType>
     347</xs:schema>
     348}}}
     349
     350From a high-level view, the TileSetDefinition is essentially the BaseMapDefinition portion of the MapDefinition schema ripped out and put into its own resource, but with the following unique characteristics:
     351
     352 * Ability to have its own custom tile image sizes
     353 * Ability to have its own tile image format
     354
     355As a forward looking measure a TileSetDefinition supports the concept of a Tile Provider (think FDO for tile sets). For the short term to see this feature realized there will only be one Tile Provider (the existing MapGuide-designated directory on the file system), but the schema provides a name/value pair mechanism for specifying connection parameters, that allows for MapGuide to support tile storage/retrieval from sources other than the MapGuide-designated file system (eg. MBTiles)
     356
     357== APIs to support tile sets ==
     358
     359The MgTileService will get new APIs to work with TileSetDefinitions:
     360
     361{{{
     362class MG_MAPGUIDE_API MgTileService : public MgService
     363{
     364PUBLISHED_API:
     365    /// Clears the entire tile cache for the given map.  Tiles for all base
     366    /// map layer groups and finite scales will be removed.
     367    ///
     368    virtual void ClearCache(MgResourceIdentifier* tileSetDefinition);
     369
     370    /// Marks a given tile cache as locked. A locked tile cache will prevent resource
     371    /// content changes in up-stream dependent resources like Layer Definitions and
     372    /// Feature Sources
     373    ///
     374    virtual void LockCache(MgResourceIdentiifer* tileSetDefinition);
     375
     376    /// Marks a given tile cache as un-locked. An un-locked tile cache can be invalidated
     377    /// by any resource content changes in up-stream dependent resources like Layer Definitions
     378    /// and Feature Sources.
     379    ///
     380    virtual void UnlockCache(MgResourceIdentiifer* tileSetDefinition);
     381};
     382}}}
     383
     384Or if we want to combine this with the HTTP 304 cacheability support
     385{{{
     386class MG_MAPGUIDE_API MgTileService : public MgService
     387{
     388PUBLISHED_API:
     389    ///
     390    ///
     391    virtual void ClearCache(MgResourceIdentifier* tileSetDefinition);
     392
     393    /// Marks a given tile cache as locked. A locked tile cache will prevent resource
     394    /// content changes in up-stream dependent resources like Layer Definitions and
     395    /// Feature Sources
     396    ///
     397    virtual void LockCache(MgResourceIdentiifer* tileSetDefinition);
     398
     399    /// Marks a given tile cache as un-locked. An un-locked tile cache can be invalidated
     400    /// by any resource content changes in up-stream dependent resources like Layer Definitions
     401    /// and Feature Sources.
     402    ///
     403    virtual void UnlockCache(MgResourceIdentiifer* tileSetDefinition);
     404
     405    /// Returns the timestamp of when the tile for the specified map/group/row/col/scale was generated. Returns NULL if no such tile exists
     406    ///
     407    MgDateTime* GetTileCreationDate(MgResourceIdentifier* resource, CREFSTRING baseMapLayerGroupName, INT32 tileColumn, INT32 tileRow, INT32 scaleIndex);
     408
     409    /// Returns the specified base map tile for the given map.  If a cached tile
     410    /// image exists it will return it, otherwise the tile is rendered and added
     411    /// to the cache.
     412    ///
     413    MgTile* GetTile(MgResourceIdentifier* resource, CREFSTRING baseMapLayerGroupName, INT32 tileColumn, INT32 tileRow, INT32 scaleIndex);
     414};
     415}}}
     416
     417Notice that no new GetTile overload is introduced (assuming this feature is separate from the HTTP 304 cacheability support). This is because tile access requires the same set of parameters, except that instead of a Map Definition, we can specify a TileSetDefinition instead. On the implementation side, the GetTile methods need to support TileSetDefinition as a valid resource type and be able to resolve the physical tile storage location to fetch and store tiles in from the TileSetDefinition. For a first cut implementation, it is safe to just hard-code this logic to resolve to the same physical tile cache directory that the existing MapGuide tile cache code uses.
     418
     419== Tile Set Locking ==
     420
     421To prevent accidental invalidation due to editing upstream dependent resources, the MgTileService will support the concept of locking and unlocking TileSetDefinition resources.
     422
     423When a TileSetDefinition is saved for the first time as a new resource it is in a default unlocked state. An unlocked Tile Set is susceptible to all the usual tile cache invalidation mechanisms:
     424
     425 * The TileSetDefinition is overwritten
     426 * An upstream Layer Definition or Feature Source has changed
     427
     428When a TileSetDefinition is locked with the MgTileService::LockCache() API, the above scenarios will be prevented via throwing the appropriate MgException (class type TBD) on saving of resource content on a TileSetDefinition or any of its upstream resource types. The exception thrown should carry the resource id of the downstream TileSetDefinition that would be affected by this operation so it can be relayed to the user or client application that they can't save this resource because it would invalidate a locked tile set.
     429
     430The act of locking a TileSetDefinition is basically saying that this tile cache and all upstream resources are "read-only" and its tile cache can not be invalidated unless explicitly unlocked.
     431
     432The act of having a TileSetDefinition unlocked is saying that this tile cache could be invalidated at any time.
     433
     434The actual implementation details are still TBD. The tentative idea would be to leverage the existing server Resource Service APIs (ie. DBXML) to store the tile set lock information, relieving us of synchronization/multi-threaded concerns to read/write lock state (Further discussion required on whether this is a good idea)
     435
     436== Map Definition Schema changes ==
     437
     438To retain the additive qualities of previous schema revisions. The Map Definition schema will simply support a new optional TileSetSource element which is simply a reference to a TileSetDefinition
     439
     440{{{
     441       <xs:element name="BaseMapDefinition" minOccurs="0">
     442        ...
     443       </xs:element>
     444       <xs:element name="TileSetSource" minOccurs="0">
     445        <xs:annotation>
     446          <xs:documentation>A reference to the tile set source to use</xs:documentation>
     447        </xs:annotation>
     448        <xs:complexType>
     449          <xs:sequence>
     450            <xs:element name="ResourceId" type="xs:string">
     451              <xs:annotation>
     452                <xs:documentation>ResourceId of the TileSetDefinition</xs:documentation>
     453              </xs:annotation>
     454            </xs:element>
     455          </xs:sequence>
     456        </xs:complexType>
     457      </xs:element>
     458}}}
     459
     460Only up to 1 TileSetSource can be defined for any Map Definition as having multiple TileSetSources presents complex situations like how to address disparity of scale ranges between different TileSetSources.
     461
     462Because TileSetDefinitions are bound to a specific coordinate system, the CoordinateSystem element of the Map Definition will no longer be the single point of truth when determining if dynamic layers in a map require re-projection. Instead if a Map Definition references a TileSetDefinition, its CoordinateSystem element takes precedence over the one defined in the Map Definition. This requirement can be relaxed/lifted if/when we support raster re-projection of existing rendered tiles (Discussion required: Is this feasible?)
     463
     464In terms of parsing into a MgMap at runtime, the TileSetSource and BaseMapDefinition are mutually exclusive. In the event that both are defined (because the XML schema allows for it), the BaseMapDefinition element "wins".
     465
     466=== MgMap changes ===
     467
     468Because Map Definitions are parsed into MgMap objects at runtime, we need to update MgMap and its associated classes (MgLayer/MgLayerGroup) to also take into account support for TileSetDefinitions
     469
     470The GetMapSRS() method will stay the same, but its behaviour will change. It will now return the CoordinateSystem of the referenced TileSetDefinition if one exists.
     471
     472Depending on which one is defined (or which one "wins" under our rules of element precendence), the finite scale list will be initialized from either the BaseMapDefinition element or the referenced TileSetDefinition
     473
     474The MgLayerGroupType class will introduce a new constant
     475{{{
     476class MgLayerGroupType
     477{
     478PUBLISHED_API:
     479    /// Specifies that the layer is a base map layer from a TileSetDefinition resource
     480    ///
     481    static const INT32 BaseMapFromTileSet = 3;
     482}
     483}}}
     484
     485The MgLayerGroup will have a new API to return the TileSetDefinition resource id where this group comes from
     486{{{
     487class MG_PLATFORMBASE_API MgLayerGroup : public MgNamedSerializable
     488{
     489PUBLISHED_API:
     490    /// Returns the resource id of the Tile Set Definition that this group originates from.
     491    /// Returns NULL if GetLayerGroupType() is not MgLayerGroupType::BaseMapFromTileSet
     492    ///
     493    MgResourceIdentifier* GetTileSetDefinition();
     494}
     495}}}
     496
     497== Client application logic changes to support TileSetDefinition resources ==
     498
     499The beautiful thing about this new feature is that from the client application perspective, the calling interface remains the same. The only difference is that the Map Definition can now also be a TileSetDefintion, and client applications should know which resource type to use with the new API in MgLayerGroup
     500
    136501= WMS =
    137502
    … …  
    148513
    149514[wiki:GoogleSoC2007#TileMapServicePublishing TMS]   
    150