Changes between Version 1 and Version 2 of FDORfc19

Timestamp:

Apr 30, 2008, 12:28:25 PM (16 years ago)

Author:

gregboone

Comment:

--

FDORfc19

@@ -23 +23 @@
 == Overview ==
 
+This document contains a design for software and file version handling strategies for FDO. The versions dealt by this document are:
+
+        • Version of FDO software[[br]]
+        • FDO provider versions[[br]]
+        • XML configuration file versions[[br]]
+
 == Motivation ==
 
-== Implications ==
+In FDO 3.x, a number of issues arose regarding the handling of Provider versions. There were a number of concerns as to how these versions should be handled going forward. The main purpose of this design is to resolve these issues so that customers and client applications can easily upgrade to new FDO software releases as they become available.
+
+== Requirements ==
+
+=== FDO and Provider Versions ===
+
+The main requirement is to ensure that, when a client asks FDO to load a particular provider, the version that is loaded is compatible with the current FDO.
+
+When client stores connection information, it typically includes the provider name. This name is tied to a particular provider version.  However, this is problematic if the next major revision of the client software loads the provider for a particular connection, using the provider name stored in the client specific file.
+
+The above would attempt to retrieve the 3.0 version of the provider, which is likely binary incompatible with the next major revision of the client, which would be compatible with FDO 4.0. This can be worked around, by changing the version part of the provider name from 3.0 to 4.0, before requesting to load it. However, it would be good to streamline this so that FDO clients do not have to keep track of what all the current provider versions are. 
+
+In FDO 3.1, work was done to address this by allowing clients to omit the version parts of the provider name when requesting to load it. In this case, the latest registered version of the provider is retrieved. However, there is no guarantee that this version of the provider is binary-compatible with the current version of FDO.
+
+FDO consists of a number of core libraries (e.g. Fdo.dll, !FdoCommon.dll, etc ) plus a number of providers. Each provider handles a different type of data. The list of providers is extensible (customers can write their own). Each provider must be registered in the provider registry, in order for the FDO core to be able to find it. The provider registry can contain multiple versions of the same provider. 
+
+Clients load providers via the FDO Feature Access Manager (FAM). The client supplies the provider name and the FAM finds the provider in the registry.  The provider name can contain the name parts plus optional version number (e.g. "OSGeo.SDF" (no version), "OSGeo.SDF.3" (name plus major version), "OSGeo.SDF.3.1" (name plus major and minor version)). 
+
+Currently, the FAM retrieves the latest registered version of the requested provider. For example, if the registry contains the following providers:
+
+        OSGeo.SDF.3.0[[br]]
+        OSGeo.SDF.3.1[[br]]
+        OSGeo.SDF.3.2[[br]]
+        OSGeo.SDF.4.0[[br]]
+        OSGeo.SDF.5.0[[br]]
+
+the following table shows which provider is returned by the FAM, depending on the requested name.
+
+||'''Client Requested'''||'''FAM Returns'''||
+||OSGeo.SDF||OSGeo.SDF.5.0||
+||OSGeo.SDF.3||OSGeo.SDF.3.2||
+||OSGeo.SDF.4||OSGeo.SDF.4.0||
+||OSGeo.SDF.3.1||OSGeo.SDF.3.1||
+
+The client decides how precise the search is by how many version parts are included in the requested provider name.
+
+In the above example, "OSGeo.SDF.5.0" is returned when "OSGeo.SDF" is requested. However, suppose that this provider is compatible with FDO version 5.0 and the version of FDO, used by the client, is 4.0. When there is a major version number difference then there is a good chance that the provider is binary incompatible with FDO, meaning that a crash or exception will occur when the provider is loaded. 
+
+=== Config File Versions ===
+
+FDO provides the ability to serialize datastore contents to the FDO XML format. This includes feature schemas, spatial contexts, logical/physical mappings (Schema Overrides) and the features themselves.  FDO XML format data can be stored in a file. The FDO XML format can change between major FDO revisions so there are versioning issues surrounding these files. These are mainly compatibility issues that arise when a file is written by one version of FDO and read by another.
+
+In order for a particular version of FDO to determine if it can read a file, it needs to know what version of FDO created the file. Therefore, the file must contain this version information. The following sections discuss the versioning requirements for each type of serializable object. 
+
+==== Feature Schemas ====
+
+Feature Schemas are currently written to XML without FDO version information. This version information needs to be added, going forward, in case the XML format for Feature Schemas changes in future. When a particular version of FDO reads a feature schema from a config file, it may need to know this version in order to read it properly.
+
+==== Spatial Contexts ====
+
+Similarly, Spatial Contexts are written to XML without FDO version information. This version information need to be added, going forward.
+
+==== Schema Overrides ====
+
+Schema Overrides are written to XML with FDO versioning information. The overrides include the name, major version and minor version of the provider that wrote them.
+
+The main requirement is to make sure that, when Schema Overrides are read from XML, they are parsed into objects that are binary compatible with the current FDO. 
+
+When FDO reads a Schema Override Set from XML, it delegates the reading to the provider that wrote the overrides. Schema Overrides are provider-specific, so only the provider knows how to read or write them. Currently, FDO chooses the earliest version of the provider that is equal to or greater than the version of the Schema Override set. However, it is possible for the chosen provider to be binary-incompatible with the current FDO. This means that the provider may either fail to be instantiated or it may parse the Schema Overrides into objects not compatible with the current FDO.
+
+==== Features ====
+
+Features are written to XML without FDO version information. This version information needs to be added, going forward.
+
+== Specification ==
+
+The following describes the client-visible changes proposed by this design. 
+
+=== FDO API ===
+
+==== FdoFeatureAccessManager::!GetFeatureDataObjectsVersion ====
+
+When this function is called it will return the version of the feature data objects specification that the currently loaded Feature Access Manager supports. The version number string has the form [!VersionMajor].[!VersionMinor].[!BuildMajor].[!BuildMinor].
+
+The function signature shall be:
+
+{{{
+FDO_API static FdoString* GetFeatureDataObjectsVersion();
+}}}
+
+==== FdoConnectionManager::!CreateConnection() ====
+
+When this function is passed a provider name without version part (e.g. "OSGeo.SDF" ), it returns a connection object from the latest registered version of that provider.
+
+With this proposal, the provider chosen will be the provider with the latest binary-compatible !FeatureDataObjects version. 
+
+The connection object will be instantiated from the chosen provider. If there is no compatible registered provider then !CreateConnection will return NULL.
+
+For the above provider searches, the provider's !FeatureDataObjects version will be used instead of the provider version itself. The reason for this is that the !FeatureDataObjects version is the one that corresponds to the version of the FDO core. 
+
+The provider search behaviour is unchanged when the requested provider name contains version parts. In this case, the latest provider matching these version parts is chosen. However, in these cases, the client assumes the risk that the returned connection object may be incompatible with the current FDO. 
+
+It is expected that most clients will pass in a provider name without version parts, since they would usually be interested in retrieving a binary-compatible version of the requested provider. 
+
+A provider is considered to be binary-compatible with the current FDO if the specified parts of the provider's !FeatureDataObjectsVersion match those of the current version of FDO. For example, if the !FeatureDataObjectsVersion has 2 parts, then these must match the 1st 2 parts of the current FDO's version. The following table provides some concrete examples:
+
+||'''Provider !FeatureDataObjectsVersion'''||'''Compatible Versions of FDO'''||'''Incompatible FDO Versions'''||
+||3||3.0.0.0||2.0.0.0
+||||3.1.0.0||4.0.0.0||
+||||3.2.0.1||4.1.0.0||
+||||||||
+||3.0||3.0.0.0||2.0.0.0
+||||3.0.1.0||3.1.0.0||
+||||||3.2.0.0||
+||||||||
+||3.2||3.2.0.0||3.1.0.0
+||||3.2.1.0||3.1.1.0||
+||||||3.3.0.0||
+||||||||
+||3.1.0.1||3.1.0.1||3.1.0.0
+||||||3.1.0.2||
+||||||3.1.2||
+
+The 4th example is for illustration and would not likely happen in the real world. It is unlikely that versions as similar as 3.1.0.1 and 3.1.0.2 would be incompatible.
+
+==== !FdoProviderNameTokens ====
+
+This class will be modified to support the removing of version parts from a provider name.
+
+''!ToString (new)''
+
+This function converts this tokenize provider name back to a string:
+
+{{{
+FdoStringP FdoProviderNameTokens::ToString( bool includeVersion = true )
+}}}
+
+When includeVersion is true, the full provider name is returned. When includeVersion is false, the version parts are omitted from the returned name.
+
+For Example, the following removes the version parts from a provider name:
+
+{{{
+FdoStringP srcName = L"OSGeo.SDF.3.1";
+FdoStringP destName;
+FdoProviderNameTokensP tokens = FdoProviderNameTokens::Create( srcName );
+destName = tokens->ToString( false );
+assert ( destName == L"OSGeo.SDF" );
+}}}
+
+''!GetNameTokens (modified)''
+
+For consistency, the !GetNameTokens signature will change to the following:
+
+
+{{{
+FdoStringsP FdoProviderNameTokens::GetNameTokens( bool includeVersion = true )
+}}}
+
+includeVersion is a new parameter. When false, the returned string collection will not include the version parts.
+
+For Example, the following removes the version parts from a provider name (note that a bit more work is required than in the above !ToString example):
+
+
+{{{
+FdoStringP srcName = L"Autodesk.Oracle.3.1";
+FdoStringP destName;
+FdoProviderNameTokensP tokens = FdoProviderNameTokens::Create( srcName );
+GisStringsP nameTokens = tokens->GetNameTokens( false  );
+destName = tokens->ToString( L"." );
+assert ( destName == L"Autodesk.Oracle" );
+}}}
+
+==== IProviderRegistry::!GetProviders (modified) ====
+
+!GetProviders will be enhanced to allow a client to determine if a particular provider is compatible with the current FDO:
+
+{{{
+FdoProviderCollection* IProviderRegistry::GetProviders( bool compatibleOnly = false)
+}}}
+
+compatibleOnly is a new parameter. When false, all registered providers are returned. When true, only providers compatible with the current FDO are returned. 
+
+For Example, the following code checks to see if a particular provider is compatible with the currently running FDO:
+
+{{{
+FdoPtr<IProviderRegistry> registry = FeatureAccessManager::GetProviderRegistry();
+FdoString* providerName = L"OSGeo.SDF.3.1";
+FdoPtr<FdoProviderCollection> providers = registry->GetProviders(true);
+bool isCompatible = (providers->IndexOf(providerName) >= 0);
+}}}
+
+==== FdoProvider::!GetIsCompatible (new) ==== 
+
+IProviderRegistry::!GetProviders implementations can use this function to test whether a particular provider is compatible. The signature is:
+
+{{{
+bool FdoProvider::!GetIsCompatible()
+}}}
+
+It returns true if this provider is compatible with the currently running FDO, and false otherwise.
+
+For Example, !GetIsCompatible is used to test the provider's compatibility:
+
+{{{
+FdoPtr<IProviderRegistry> registry = FeatureAccessManager::GetProviderRegistry();
+FdoString* providerName = L"OSGeo.SDF.3.1";
+FdoPtr<FdoProviderCollection> providers = registry->GetProviders(false);
+FdoInt32 ix = providers->IndexOf(providerName);
+FdoPtr<FdoProvider> provider = providers->GetItem( ix );
+bool isCompatible = provider->GetIsCompatible();
+}}}
+
+==== FdoPhysicalSchemaMappingCollection::!ReadXml (modified) ====
+
+When this function encounters a Schema Override Set, it searches the provider registry for the earliest provider whose version is equal or greater than the Schema Override Set's version. It then invokes that provider to read the Schema Override Set.
+
+With this proposal, the latest provider that is binary-compatible with the current FDO will be chosen. 
+
+It is possible that the chosen provider will be too old to read the given Schema Override Set. When the provider reads the Schema Override set, it may encounter a construct that it doesn't understand. If this happens, the action taken depends on the current !ErrorLevel passed to !ReadXml, via the flags parameter:
+
+High – an error is reported[[br]]
+Other – the construct is ignored[[br]] 
+
+=== Feature Schemas ===
+
+The fdo:version and fdo:minimumVersion elements will also be allowed on the <xs:schema> element. Both attributes are optional.
+When the FDO XML Schema writer writes a single schema, it does not wrap the schema in an <fdo:Datastore> element. This is done to maximize compatibility with external applications. In this case, the document root element is <xs:schema>. Therefore, the schema writer will add the versioning attributes to the <xs:schema> element, in this case.
+
+In order to support the required schema override change, the !FdoFeatureSchema class will be modified to include a !GetFeatureDataObjectsVersion function as well as a !GetMinimumFeatureDataObjectsVersion function.
+
+
+{{{
+class FdoFeatureSchema : 
+    public FdoSchemaElement,
+    public FdoXmlSerializable
+{
+public:
+    /// \brief
+    /// Gets the version of FDO that created the Feature Schema
+    /// 
+    /// \return
+    /// Returns the Feature Data Objects version as an FdoString.
+    /// The FDO version must conform to the format:
+    /// "[VersionMajor].[VersionMinor].[BuildMajor].[BuildMinor]".
+    /// 
+    FDO_API virtual FdoString* GetFeatureDataObjectsVersion();
+
+    /// \brief
+    /// Gets the earliest version of FDO that can possibly 
+    /// read the Schema Mapping
+    /// 
+    /// \return
+    /// Returns the Feature Data Objects version as an FdoString.
+    /// The FDO version must conform to the format:
+    /// "[VersionMajor].[VersionMinor].[BuildMajor].[BuildMinor]".
+    /// 
+    FDO_API virtual FdoString* GetMinimumFeatureDataObjectsVersion();
+…
+…
+};
+}}}
+
+=== Schema Overrides ===
+
+The Schema Overrides version is already stored in the provider attribute of the <!SchemaMapping> tag. The following new attribute (for the <!SchemaMapping> tag ) will be defined:
+
+        minimumVersion
+
+Its behaviour is identical to that of the general fdo:minimumVersion.
+
+The following updated !FdoOverride.xsd definitions reflect the proposed changes above.
+
+
+{{{
+<xs:schema>
+...
+...
+    <xs:complexType name="PhysicalSchemaMappingType" abstract="true">
+        <xs:complexContent>
+            <xs:extension base="SchemaMappingElement">
+                <xs:attribute name="provider" type="ProviderType" use="required"/>
+                <xs:attribute ref="fdo:minimumVersion" use="optional"/>
+            </xs:extension>
+        </xs:complexContent>
+    </xs:complexType>
+...
+...
+</xs:schema>
+}}}
+
+
+In order to support the required schema override change, the !FdoPhysicalSchemaMapping and !FdoXmlSchemaMapping classes will be modified to include a !GetMinimumProviderVersion function.
+
+
+{{{
+class FdoPhysicalSchemaMapping :
+        public FdoPhysicalElementMapping,
+        public FdoXmlSerializable
+{
+public:
+    /// \brief
+    /// Gets the earliest version of Provider that can possibly 
+    /// read the Schema Mapping
+    /// 
+    /// \return
+    /// Returns the version as a constant wchar_t*.
+    /// The version must conform to the format:
+    /// "[VersionMajor].[VersionMinor].[BuildMajor].[BuildMinor]".
+    /// 
+    FDO_API virtual FdoString* GetMinimumProviderVersion () = 0;
+…
+…
+};
+
+/// \brief
+/// FdoXmlSchemaMapping specifies overrides for translating a feature schema 
+/// between FDO and GML.
+class FdoXmlSchemaMapping :
+        public FdoPhysicalSchemaMapping
+{
+public:
+    /// \brief
+    /// Gets the earliest version of Provider that can possibly 
+    /// read the Schema Mapping
+    /// 
+    /// \return
+    /// Returns the version as a constant wchar_t*.
+    /// The version must conform to the format:
+    /// "[VersionMajor].[VersionMinor].[BuildMajor].[BuildMinor]".
+    /// 
+    FDO_API FdoString* GetMinimumProviderVersion ();
+…
+…
+};
+}}}
+
+=== Managed API ===
+
+Here are the Managed API wrapper definitions for the Unmanaged API described above.
+
+
+{{{
+/// \brief
+/// The FeatureAccessManager class manages a registry of feature 
+/// providers stored in the registry and provides support for 
+/// dynamic discovery and binding to registered feature providers
+public __sealed __gc class FeatureAccessManager 
+{
+public:
+…
+…
+    /// \brief
+    /// Gets the version of the feature data objects 
+    /// specification the Feature Access Manager conforms to. 
+    /// The version number string has the form 
+    /// [VersionMajor].[VersionMinor].[BuildMajor].[BuildMinor].
+    /// 
+    /// \return
+    /// Returns the Feature Data Objects version as a constant wchar_t*.
+    /// 
+    static System::String* GetFeatureDataObjectsVersion();
+…
+…
+};
+
+/// \brief
+/// A provider name tokenized into its company, name and version parts.
+public __gc class ProviderNameTokens : public Disposable
+{
+public:
+…
+…
+    /// \brief
+    /// Gets all of the tokens in this provider name
+    /// 
+    /// \return
+    /// Returns the collection of tokens. 
+    ///   Element 0 is the company
+    ///   Element 1 is the unqualified name
+    ///   the rest of the elements are the individual 
+    /// parts of the version number.
+    /// 
+    System::String* GetNameTokens()[];
+
+    /// \brief
+    /// Gets all of the tokens in this provider name
+    /// 
+    /// \param includeVersion 
+    /// Input a boolean flag indicating that the version information
+    /// should be returned as a part of the provider name. When 
+    /// includeVersion is true, the full provider name is returned. 
+    /// When includeVersion is false, the version parts are omitted 
+    /// from the returned name.
+    /// 
+    /// \return
+    /// Returns the collection of tokens. 
+    ///   Element 0 is the company
+    ///   Element 1 is the unqualified name
+    ///   the rest of the elements are the individual 
+    /// parts of the version number.
+    /// 
+    System::String* GetNameTokens( Boolean includeVersion)[];
+
+    /// \brief
+    /// Converts the tokenized provider name back to a string.
+    /// 
+    /// \param includeVersion 
+    /// Input a boolean flag indicating that the version information
+    /// should be returned as a part of the provider name. When 
+    /// includeVersion is true, the full provider name is returned. 
+    /// When includeVersion is false, the version parts are omitted 
+    /// from the returned name.
+    /// 
+    /// \return
+    /// Returns the tokenized provider name.
+    /// 
+    System::String* ToString( Boolean includeVersion );
+…
+…
+};
+
+/// \brief
+/// The IProviderRegistry interface supports registering, 
+/// un-registering, and enumerating registered feature providers. 
+/// \note
+/// This is not the Windows registry.
+public __gc __interface IProviderRegistry : public System::IDisposable
+{
+public:
+…
+…
+
+    /// \brief
+    /// Gets a read only collection of information describing 
+    /// each of the registered feature providers. 
+    /// 
+    /// \return
+    /// Returns an an instance of ProviderCollection.
+    ProviderCollection* GetProviders();
+
+    /// \brief
+    /// Gets a read only collection of information describing 
+    /// each of the registered feature providers. 
+    /// 
+    /// \param compatibleOnly 
+    /// When false, all registered providers are returned. 
+    /// When true, only providers compatible with the current FDO are returned. 
+    ///
+    /// \return
+    /// Returns an an instance of ProviderCollection.
+    ProviderCollection* GetProviders( Boolean compatibleOnly );
+…
+…
+};
+
+/// \brief
+/// ProviderRegistry supports registering, un-registering, and 
+/// enumerating registered feature providers. 
+/// \note
+/// This is not the Windows registry.
+public __gc class ProviderRegistry : 
+public Disposable, 
+public IProviderRegistry
+{
+public:
+…
+…
+
+    /// \brief
+    /// Gets a read only collection of information describing 
+    /// each of the registered feature providers. 
+    /// 
+    /// \return
+    /// Returns an an instance of ProviderCollection.
+    ProviderCollection* GetProviders();
+
+    /// \brief
+    /// Gets a read only collection of information describing 
+    /// each of the registered feature providers. 
+    /// 
+    /// \param compatibleOnly 
+    /// When false, all registered providers are returned. 
+    /// When true, only providers compatible with the current FDO are returned. 
+    ///
+    /// \return
+    /// Returns an an instance of ProviderCollection.
+    ProviderCollection* GetProviders ( Boolean compatibleOnly );
+…
+…
+};
+
+/// \brief
+/// Provides information about a feature provider, including name, description, 
+/// library, and version information.
+public __sealed __gc class Provider : public Disposable
+{
+public:
+…
+…
+    /// \brief
+    /// Gets a boolean flag indicating if the provider is compatible with the 
+    /// currently running version of FDO. 
+    /// 
+    /// \return
+    /// Returns returns true if this provider is compatible with the currently 
+    /// running FDO, and false otherwise.
+    /// 
+    __property System::Boolean get_IsCompatible();
+ …
+ …
+};
+
+/// \brief
+/// The FdoFeatureSchema class derives from FdoSchemaElement.
+/// A feature schema contains all of the classes and relationships
+/// that make up a particular data model. The FdoFeatureSchema class 
+/// can be used to either create a new schema or to browse the schema end of a
+/// connection. In the later case, the FdoFeatureSchema instance is created by 
+/// the DescribeSchema command. In this case the schema objects have 
+/// additional properties, such as coordinate system definitions that 
+/// can be useful to the
+/// application when placed in context with the schema objects.
+public __gc class FeatureSchema : 
+public SchemaElement, 
+public IXmlSerializable
+{
+public:
+…
+…
+    /// \brief
+    /// Gets the version of FDO that created the Feature Schema
+    /// 
+    /// \return
+    /// Returns the Feature Data Objects version as an FdoString.
+    /// The FDO version must conform to the format:
+    /// "[VersionMajor].[VersionMinor].[BuildMajor].[BuildMinor]".
+    /// 
+    __property System::String* get_FeatureDataObjectsVersion ();
+
+    /// \brief
+    /// Gets the earliest version of FDO that can possibly 
+    /// read the Schema Mapping
+    /// 
+    /// \return
+    /// Returns the Feature Data Objects version as an FdoString.
+    /// The FDO version must conform to the format:
+    /// "[VersionMajor].[VersionMinor].[BuildMajor].[BuildMinor]".
+    /// 
+    __property System::String* get_MinimumFeatureDataObjectsVersion ();
+…
+…
+};
+
+
+/// \brief
+/// PhysicalSchemaMapping is the base class of all Schema Override sets. 
+/// Each instance contains the overrides for a particular Feature Schema 
+/// and FDO Provider. Each FDO Provider, that allows Schema Overrides, must 
+/// create a sub-class of this class. This sub-class must implement the 
+/// overrides that are specific to the provider. The Provider can also add 
+/// support, for serializing to an XML document, by overriding the functions 
+/// inherited from  FdoXmlSerializable.
+public __gc class PhysicalSchemaMapping : public PhysicalElementMapping
+{
+public:
+…
+…
+    /// \brief
+    /// Gets the earliest version of the Provider that can 
+    /// possibly read the Schema Mapping
+    /// 
+    /// \return
+    /// Returns the Provider version as an FdoString.
+    /// The version must conform to the format:
+    /// "[VersionMajor].[VersionMinor].[BuildMajor].[BuildMinor]".
+    /// 
+    __property System::String* get_MinimumProviderVersion();
+…
+…
+};
+
+
+/// \brief
+/// FdoXmlSchemaMapping specifies overrides for translating 
+/// a feature schema between FDO and GML.
+public __gc class XmlSchemaMapping : public PhysicalSchemaMapping
+{
+public:
+…
+…
+
+    /// \brief
+    /// Gets the earliest version of the Provider that can possibly 
+    /// read the Schema Mapping
+    /// 
+    /// \return
+    /// Returns the Provider version as an FdoString.
+    /// The version must conform to the format:
+    /// "[VersionMajor].[VersionMinor].[BuildMajor].[BuildMinor]".
+    /// 
+    __property System::String* get_MinimumProviderVersion();
+…
+…
+};
+}}}
+
+
+
 
 == Test Plan ==