Context Navigation

Changes between Version 4 and Version 5 of FdoEnhancedSchemaNameSupport

Timestamp:: Oct 18, 2007, 2:51:51 PM (17 years ago)
Author:: gregboone
Comment:: --

Legend:

: Unmodified
: Added
: Removed
: Modified

FdoEnhancedSchemaNameSupport

-              v4
+              v5
 The FDO API provides functions for translating FDO Feature Schemas to and from the !OpenGeospatial GML format. These functions are provided to satisfy 4 main use cases:
 • Export/Import: to provide a text based export format for FDO Feature Schemas. This export format actually covers other types of objects such as Spatial Contexts, Schema Overrides and Features. However, only Feature Schemas are pertinent to this document.
 • Schema Exchange: Allow exchange of schemas between FDO and external GML-based applications.
 • WFS Provider: used by the FDO WFS Provider to translated GML schemas, provided by the connected WFS, to FDO Schemas.
 • Publish as WFS: Allow FDO accessible data to be published via a WFS. This is the opposite of the previous use case.
+        • Export/Import: to provide a text based export format for FDO Feature Schemas. This export format actually covers other types of objects such as Spatial Contexts, Schema Overrides and Features. However, only Feature Schemas are pertinent to this document.
+        • Schema Exchange: Allow exchange of schemas between FDO and external GML-based applications.
+        • WFS Provider: used by the FDO WFS Provider to translated GML schemas, provided by the connected WFS, to FDO Schemas.
+        • Publish as WFS: Allow FDO accessible data to be published via a WFS. This is the opposite of the previous use case.
 One of the big challenges, in translating schemas between FDO and GML, is the converting of schema names.  In order to support the above use cases, these conversions must satisfy the following general requirements:
 • round trip fidelity. If the schema is translated from GML to FDO to GML, the schema name in the resulting GML schema document must be the same as in the original. Similarly, the name must not change when translated from FDO to GML to FDO.
 • name uniqueness must be preserved. Different GML schemas must get different FDO schema names when read into FDO. Conversely different FDO schemas must get different GML schema names when written to GML. If name uniqueness is not preserved, schemas will be unexpectedly merged on read or write.
+        • round trip fidelity. If the schema is translated from GML to FDO to GML, the schema name in the resulting GML schema document must be the same as in the original. Similarly, the name must not change when translated from FDO to GML to FDO.
+        • name uniqueness must be preserved. Different GML schemas must get different FDO schema names when read into FDO. Conversely different FDO schemas must get different GML schema names when written to GML. If name uniqueness is not preserved, schemas will be unexpectedly merged on read or write.
 The structure of schema names differs greatly in either format:
 • in FDO, a schema name is a free-form name, containing any character except '.' and ':'. Names tend to be short; more detailed information is typically kept in the schema description.
 • in GML, the schema name must be a valid URI. Most current FDO schema names are valid URI's. However, most GML schema names tend to conform to the http scheme (see glossary), as seen in the following example. FDO Schema names would tend to not fit the http scheme.
+        • in FDO, a schema name is a free-form name, containing any character except '.' and ':'. Names tend to be short; more detailed information is typically kept in the schema description.
+        • in GML, the schema name must be a valid URI. Most current FDO schema names are valid URI's. However, most GML schema names tend to conform to the http scheme (see glossary), as seen in the following example. FDO Schema names would tend to not fit the http scheme.
 A typical example might be a Roads schema defined by the municipality "MyCity". The FDO schema might simply be "Roads". However, the GML schema name might look something like this:
+        http://www.mycity.on.ca/departments/transportation/Roads
+where the schema name is qualified by the owning organization. This makes it difficult to perform the schema name conversion in a way that satisfies the abovementioned requirements.
+The FDO API provides a number of methods to ensure round trip fidelity and preservation of schema name uniqueness. However, these methods are cumbersome for some of the abovementioned use cases. This document looks at alternatives for making schema name translation easier when performed through the FDO API.
+=== Current API ===
+Feature Schema translation is provided by 2 functions on !FdoFeatureSchemaCollection:
+        • !ReadXml() converts GML schemas to FDO[[br]]
+        • !WriteXml() converts FDO schemas to GML (!WriteXml() is also present on !FdoFeatureSchema to allow the writing of individual schemas).
+Both of the above functions take optional !FdoXmlFlags parameters, which control how the translation is performed.
+The following sub-sections look at the various schema name translation options currently provided:
+==== Default Translation ====
+===== FDO to GML =====
+When no !FdoXmlFlags are specified, the FDO schema name is translated by prepending a default osgeo-defined schema prefix (http://fdo.osgeo.org/schemas/feature/) to the schema name and escaping any characters not allowed in a URI. For example, the FDO Schema "Water Service" becomes:
+http://fdo.osgeo.org.schemas/feature/Water-x20-Service
+===== GML to FDO =====
+When no !FdoXmlFlags are specified, GML schema names are translated by dropping any http:// prefix and escaping '.' and ':' to '-dot-' and '-colon-' respectively. This means that the example schema name from the Overview:
 http://www.mycity.on.ca/departments/transportation/Roads
-where the schema name is qualified by the owning organization. This makes it difficult to perform the schema name conversion in a way that satisfies the abovementioned requirements.
-The FDO API provides a number of methods to ensure round trip fidelity and preservation of schema name uniqueness. However, these methods are cumbersome for some of the abovementioned use cases. This document looks at alternatives for making schema name translation easier when performed through the FDO API.
-=== Current API ===
-Feature Schema translation is provided by 2 functions on !FdoFeatureSchemaCollection:
-• !ReadXml() converts GML schemas to FDO[[br]]
-• !WriteXml() converts FDO schemas to GML (!WriteXml() is also present on !FdoFeatureSchema to allow the writing of individual schemas).
-Both of the above functions take optional !FdoXmlFlags parameters, which control how the translation is performed.
-The following sub-sections look at the various schema name translation options currently provided:
-==== Default Translation ====
-===== FDO to GML =====
-When no !FdoXmlFlags are specified, the FDO schema name is translated by prepending a default osgeo-defined schema prefix (http://fdo.osgeo.org/schemas/feature/) to the schema name and escaping any characters not allowed in a URI. For example, the FDO Schema "Water Service" becomes:
-http://fdo.osgeo.org.schemas/feature/Water-x20-Service
-===== GML to FDO =====
-When no !FdoXmlFlags are specified, GML schema names are translated by dropping any http:// prefix and escaping '.' and ':' to '-dot-' and '-colon-' respectively. This means that the example schema name from the Overview:
-http://www.mycity.on.ca/departments/transportation/Roads
 becomes:
 www-dot-mycity-dot-on-dot-ca/departments/transportation/Roads
+        www-dot-mycity-dot-on-dot-ca/departments/transportation/Roads
 This preserves name uniqueness but leads to a rather messy looking FDO schema name that is not easily human-readable.
 …
 When the GML schema name begins with the default schema prefix (http://fdo.osgeo.org/schemas/feature/), this whole prefix is removed from the schema name. For example:
 http://fdo.osgeo.org/schemas/Roads
+        http://fdo.osgeo.org/schemas/Roads
 simply becomes:
 Roads.
+        Roads.
 This is done to preserve round-trip fidelity.
 …
 There is also a defect which occurs when a schema name, that doesn't start with the default  prefix, is round tripped from GML to FDO to GML. The GML Schema name:
 http://www.mycity.on.ca/departments/transportation/Roads
+        http://www.mycity.on.ca/departments/transportation/Roads
 becomes:
 www-dot-mycity-dot-on-dot-ca/departments/transportation/Roads
+        www-dot-mycity-dot-on-dot-ca/departments/transportation/Roads
 in FDO. However, when written back to GML, the default prefix is still prepended to the schema name, giving:
 http: //fdo.osgeo.org/schemas/feature/www.mycity.on.ca/departments/transportation/Roads
+        http: //fdo.osgeo.org/schemas/feature/www.mycity.on.ca/departments/transportation/Roads
 Therefore, round trip fidelity is not preserved.
 …
 This method works better, for the Publish as WFS use case, than the default method. The customer can control what the GML schema name looks like and can make it reflect the URI's used by their organization. However, there is a limitation in that the same schema prefix gets applied to each schema in the schema collection. For example, if the schema contains a "Roads" and "!WaterService" schema, and the desired GML Schema names are:
         www.mycity.on.ca/departments/transportation/Roads
+        www.mycity.on.ca/departments/transportation/Roads[[br]]
         www.mycity.on.ca/departments/watersewer/WaterService
 …
 The current API handles the Export/Import and WFS Provider uses cases fairly well. However, the handling of the Schema Exchange and Publish as WFS use cases is cumbersome for the following reasons:
 • customer must usually supply a schema  prefix and must ensure that consistent prefixes are used when reading and writing the feature schemas.
 • due to a bug, round trip fidelity of schema names is not preserved when the schema name does not start with the schema prefix.
 • For a single !ReadXml() or !WriteXml() operation, there is no way to apply a different schema prefix to each schema.
 • On !ReadXml(), if the GML schema name is not prefixed by the schema prefix, a messy looking FDO schema name is generated.
+        • customer must usually supply a schema  prefix and must ensure that consistent prefixes are used when reading and writing the feature schemas.
+        • due to a bug, round trip fidelity of schema names is not preserved when the schema name does not start with the schema prefix.
+        • For a single !ReadXml() or !WriteXml() operation, there is no way to apply a different schema prefix to each schema.
+        • On !ReadXml(), if the GML schema name is not prefixed by the schema prefix, a messy looking FDO schema name is generated.
 The main requirement is to streamline the handling of these two use cases. !MapGuide currently (or plans to ) supports publishing FDO data sources through a WFS, so supporting this Publish as WFS likely takes priority over Schema Exchange. (TBD: verify !MapGuide's WFS publishing requirements).
 …
 Section 'Solution Options' below explores a number of solution options. From these options, a number of recommendations can be made:
 • It is recommended that the  Schema Name Attributes option be implemented (see 5.2 Schema Name Attributes). Although not perfect, it is the best option mentioned in this document.
 • it is not currently recommended that we allow '.' and ':' in schema names (see 5.1 Simple Default Translation). However, this option would make schema names more readable so it should be explored further to see if the potential drawbacks can be addressed.
+        • It is recommended that the  Schema Name Attributes option be implemented (see 5.2 Schema Name Attributes). Although not perfect, it is the best option mentioned in this document.
+        • it is not currently recommended that we allow '.' and ':' in schema names (see 5.1 Simple Default Translation). However, this option would make schema names more readable so it should be explored further to see if the potential drawbacks can be addressed.
 === Solution Options ===
 …
 Under this option, the url setting for !FdoXmlFlags would no longer have a default. When no !FdoXmlFlags are specified, !ReadXml() and !WriteXml() would not usually modify the schema name, meaning that the GML and FDO names would usually be identical.
 ''!WriteXml()'' would only modify the name if it is not a valid URI. In this case, the name would be adjusted by escaping any non-conforming characters.
 ''!ReadXml()'' would only modify the name if it contains any escaped characters, which it would unescape. !ReadXml() would no longer escape '.' and ':'. This has implications on the !FdoFeatureSchema object in that these two characters would have to be allowed in schema names. This can be done by removing the no '.' or ':' restriction from schema element names. However, we'd need a way to indicate when these are literal characters when they appear as qualified names. One possibility is to mandate that each component of a qualified schema element name must be enclosed in double quotes when it contains a '.' or ':'.
+        ''!WriteXml()'' would only modify the name if it is not a valid URI. In this case, the name would be adjusted by escaping any non-conforming characters.
+        ''!ReadXml()'' would only modify the name if it contains any escaped characters, which it would unescape. !ReadXml() would no longer escape '.' and ':'. This has implications on the !FdoFeatureSchema object in that these two characters would have to be allowed in schema names. This can be done by removing the no '.' or ':' restriction from schema element names. However, we'd need a way to indicate when these are literal characters when they appear as qualified names. One possibility is to mandate that each component of a qualified schema element name must be enclosed in double quotes when it contains a '.' or ':'.
 When the url !FdoXmlFlag is specified, the behaviour would be as now: !WriteXml() would tack this url prefix onto the schema name and !ReadXml() would remove it if present.
 …
 '''Pros'''
 • Simpler approach. For the default case, we don't need to worry about tacking on and stripping off prefixes.
 • FDO Schema Names derived from GML are more readable. The names can still be long but at least they won't contain the '-dot-' and '-colon-' sequences.
 • Works well for the Export/Import and Schema Exchange use cases as long as both !ReadXml() and !WriteXml() are done in FDO 3.3.0. Not applicable to the WFS Provider use case.
+        • Simpler approach. For the default case, we don't need to worry about tacking on and stripping off prefixes.
+        • FDO Schema Names derived from GML are more readable. The names can still be long but at least they won't contain the '-dot-' and '-colon-' sequences.
+        • Works well for the Export/Import and Schema Exchange use cases as long as both !ReadXml() and !WriteXml() are done in FDO 3.3.0. Not applicable to the WFS Provider use case.
 '''Cons'''
 • Backward Compatibility. For the Export/Import and Schema Exchange use cases, schema name round trip fidelity is not preserved if !WriteXml() is done from pre-Slate and !ReadXml() from FDO 3.3.0 or vice versa. The reason is that one operation will use a different name translation method from the other. This is more of a problem for Export/Import. The bug mentioned above already introduces round trip fidelity problems for the Schema Exchange use case. The backward compatibility issues could be mitigated if we could detect when one of the operations was done using pre-slate. In this case the operation done in slate would use the pre-slate name translation rules. However, this eliminates the simplicity pro since we still have to keep the old name translation algorithms around.
 • Provider compatibility. Some providers might need changes, since current code based on assumption that schema element names do not contain '.' or ':'.
 • Qualified Schema Element name compatibility. This option changes the rules for constructing qualified element names. In pre-slate, double quotes are always literals but in Slate they would be delimiters. It is unlikely that any pre-existing schema names start and end with a double quote but it is possible.
 • GML Schema names no longer guaranteed to follow the http scheme. Almost every GML schema we've seen sofar follows this URI naming scheme, so customers might complain if we don't always follow it. Therefore, this option does not work well for the Publish as WFS use case. Clients doing this sort of publishing will likely start specifying FdoXmlFlags to WriteXml to ensure that the GML Schema name follows the http scheme.
+        • Backward Compatibility. For the Export/Import and Schema Exchange use cases, schema name round trip fidelity is not preserved if !WriteXml() is done from pre-Slate and !ReadXml() from FDO 3.3.0 or vice versa. The reason is that one operation will use a different name translation method from the other. This is more of a problem for Export/Import. The bug mentioned above already introduces round trip fidelity problems for the Schema Exchange use case. The backward compatibility issues could be mitigated if we could detect when one of the operations was done using pre-slate. In this case the operation done in slate would use the pre-slate name translation rules. However, this eliminates the simplicity pro since we still have to keep the old name translation algorithms around.
+        • Provider compatibility. Some providers might need changes, since current code based on assumption that schema element names do not contain '.' or ':'.
+        • Qualified Schema Element name compatibility. This option changes the rules for constructing qualified element names. In pre-slate, double quotes are always literals but in Slate they would be delimiters. It is unlikely that any pre-existing schema names start and end with a double quote but it is possible.
+        • GML Schema names no longer guaranteed to follow the http scheme. Almost every GML schema we've seen sofar follows this URI naming scheme, so customers might complain if we don't always follow it. Therefore, this option does not work well for the Publish as WFS use case. Clients doing this sort of publishing will likely start specifying FdoXmlFlags to WriteXml to ensure that the GML Schema name follows the http scheme.
 Conclusions:
 • If we were starting from scratch, this would be a good option. However, at this stage, it is not really viable, due to backward compatibility issues, since these issues wipe out the advantages in simplicity.
+        • If we were starting from scratch, this would be a good option. However, at this stage, it is not really viable, due to backward compatibility issues, since these issues wipe out the advantages in simplicity.
 ==== Schema Name Attributes ====
 …
 An xs:schema/fdo:name attribute would be added to the  FDO XML format, to specify the equivalent FDO name for schema.
 !WriteXml() would write this attribute to the GML document.
 when present, !ReadXml() would take this attribute as the FDO schema name. Otherwise, the FDO Schema name would be generated from the GML name as is currently done.
+        !WriteXml() would write this attribute to the GML document.
+        when present, !ReadXml() would take this attribute as the FDO schema name. Otherwise, the FDO Schema name would be generated from the GML name as is currently done.
 A globalName attribute would be added to the !FdoFeatureSchema class:
 !ReadXml() would set this attribute.
 when present, !WriteXml() would use this attribute as the GML schema name. Otherwise, the GML schema name would be generated from the FDO schema name, as is currently done. Hopefully, the !GlobalName would be set to a valid URI. If not then !WriteXml() would escape any non-conforming characters. Alternatively, we could restrict the !GlobalName to be a valid URI.
+        !ReadXml() would set this attribute.
+        when present, !WriteXml() would use this attribute as the GML schema name. Otherwise, the GML schema name would be generated from the FDO schema name, as is currently done. Hopefully, the !GlobalName would be set to a valid URI. If not then !WriteXml() would escape any non-conforming characters. Alternatively, we could restrict the !GlobalName to be a valid URI.
 From a semantic standpoint, the name attribute for !FdoFeatureSchema would be unique within a particular domain (e.g.: an FDO Datastore, an !FdoFeatureSchemaCollection in an application). The !GlobalName attribute would be intended to be universally unique, or least unique among all organizations that use the feature schema.
 …
 Pros:
 • Good for Schema Exchange  use case. Customer no longer needs to ensure the same url !FdoXmlFlag is used for both !WriteXml() and !ReadXml(). For the Schema Exchange, case it also opens up the possibility of using the Short Prefix as Schema Name method, since it eliminates the problem of reconstituting the GML schema name on !WriteXml().
 • Flexible. The GML and FDO schema name correspondences can be set on a per-schema basis.
 • Still a relatively simple option. Much simpler than the Schema Overrides and Schema Namespaces options discussed below.
 • Good for Publish as WFS use cases. Publisher can set the GML name for each FDO schema explicitly.
 • GML schema name can be persisted in an FDO Datastore.
 • Alternatively, GML schema name can be set on the Feature Schema just before it is converted to GML. This can easily be done via the FDO API.
+        • Good for Schema Exchange  use case. Customer no longer needs to ensure the same url !FdoXmlFlag is used for both !WriteXml() and !ReadXml(). For the Schema Exchange, case it also opens up the possibility of using the Short Prefix as Schema Name method, since it eliminates the problem of reconstituting the GML schema name on !WriteXml().
+        • Flexible. The GML and FDO schema name correspondences can be set on a per-schema basis.
+        • Still a relatively simple option. Much simpler than the Schema Overrides and Schema Namespaces options discussed below.
+        • Good for Publish as WFS use cases. Publisher can set the GML name for each FDO schema explicitly.
+        • GML schema name can be persisted in an FDO Datastore.
+        • Alternatively, GML schema name can be set on the Feature Schema just before it is converted to GML. This can easily be done via the FDO API.
 Cons:
 • There is no guarantee that these new settings will be persisted. This limits the cases where these settings can be effective. If an FDO schema goes through the following steps:
 • Write to GML[[br]]
 • Read GML into 3rd party application[[br]]
 • Write from 3rd party application to GML[[br]]
 • Read from GML[[br]]
 the fdo:schema name will likely be lost when the schema goes through the 3rd party application.
 FDO providers would not necessarily be immediately modified to support the GML namespace !FdoFeatureSchema attribute. Therefore, the following steps:
 • Read from GML[[br]]
 • Apply schema to FDO datastore[[br]]
 • Describe schema back from datastore[[br]]
 • Write schema to GML[[br]]
 will lose the GML Schema name if the datastore's provider does not handle it. However, this particular problem can be mitigated by ensuring that the SDF and RDBMS providers support this attribute.
 This con would be applicable to the Schema Exchange and Publish as WFS use cases. Export/Import and WFS Provider would be unaffected.
 • For cases where the FDO schema name cannot be persisted in the GML document, it would be possible for the application to add it to the document just before it is read into FDO. This is possible to do but not quite straightforward since there is no simple FDO API to do this. It would have to be done via the Xerces DOM classes or by an XSL transformation.
 • There is overlap between the globalName attribute and the targetNamespace attribute on !FdoSchemaMapping (See next section), since both would represent the GML schema name. We could have a precedence rule (e.g. !FdoSchemaMapping.targetNamespace trumps !FdoFeatureSchema.globalName). However, having multiple places where the same attribute can be set makes the FDO API more complicated.
+        • There is no guarantee that these new settings will be persisted. This limits the cases where these settings can be effective. If an FDO schema goes through the following steps:
+                • Write to GML[[br]]
+                • Read GML into 3rd party application[[br]]
+                • Write from 3rd party application to GML[[br]]
+                • Read from GML[[br]]
+        the fdo:schema name will likely be lost when the schema goes through the 3rd party application.
+        FDO providers would not necessarily be immediately modified to support the GML namespace !FdoFeatureSchema attribute. Therefore, the following steps:
+                • Read from GML[[br]]
+                • Apply schema to FDO datastore[[br]]
+                • Describe schema back from datastore[[br]]
+                • Write schema to GML[[br]]
+        will lose the GML Schema name if the datastore's provider does not handle it. However, this particular problem can be mitigated by ensuring that the SDF and RDBMS providers support this attribute.
+        This con would be applicable to the Schema Exchange and Publish as WFS use cases. Export/Import and WFS Provider would be unaffected.
+        • For cases where the FDO schema name cannot be persisted in the GML document, it would be possible for the application to add it to the document just before it is read into FDO. This is possible to do but not quite straightforward since there is no simple FDO API to do this. It would have to be done via the Xerces DOM classes or by an XSL transformation.
+        • There is overlap between the globalName attribute and the targetNamespace attribute on !FdoSchemaMapping (See next section), since both would represent the GML schema name. We could have a precedence rule (e.g. !FdoSchemaMapping.targetNamespace trumps !FdoFeatureSchema.globalName). However, having multiple places where the same attribute can be set makes the FDO API more complicated.
 Conclusions
 • This is a viable option. Although it doesn't help with Schema name translation in all cases, it would still handle a lot of cases
 • Despite the overlap with GML schema overrides, this would currently be the recommended option, since it is the best one mentioned in this document.
+        • This is a viable option. Although it doesn't help with Schema name translation in all cases, it would still handle a lot of cases
+        • Despite the overlap with GML schema overrides, this would currently be the recommended option, since it is the best one mentioned in this document.