FDO RFC 34 - FDO Reader Access By Index
This page contains an change request (RFC) for the FDO Open Source project. More FDO RFCs can be found on the RFCs page.
Status
| RFC Template Version | (1.0) |
| Submission Date | April 9, 2009 |
| Last Modified | Greg Boone Timestamp |
| Author | Klain Qin |
| RFC Status | Adopted |
| Implementation Status | Implemented |
| Proposed Milestone | 3.5.0.0 |
| Assigned PSC guide(s) | Greg Boone |
| Voting History | (vote date) |
| +1 | Greg, Jason, Traian, Jackie, Orest |
| +0 | |
| -0 | |
| -1 |
Motivation
To provide faster access to data returned from feature, data and SQL readers though the FDO API.
To provide enhanced compatibility with ADO.Net Reader/Record Interfaces.
Overview
Currently, data returned in Feature readers, data or SQL readers can only be accessed using input function parameters that reflect the name of the property or column being read. In the following example, the feature reader returned from the select command has to be accessed using the GetDouble method, with an input parameter that is the feature property name. In such an implementation, there is an associated cost to determining which property is being requested. Such a cost is typically encountered in performing a lookup based on the parameter name, which inevitably results in some form of a string comparison.
FdoPtr<FdoISelect> spSelectCmd = static_cast<FdoISelect>(mConnection->CreateCommand(FdoCommandType_Select));
spSelectCmd->SetFeatureClassName (L"Foo");
FdoPtr<FdoIdentifierCollection> spIds = spSelectCmd->GetPropertyNames ();
FdoPtr<FdoIdentifier> spId = FdoComputedIdentifier::Create(L"AVG_ID", FdoPtr<FdoExpression>(FdoExpression::Parse(L"Avg(ID)")));
spIds->Add(spId);
FdoPtr<FdoIFeatureReader> spReader = spSelectCmd->Execute ();
while (spReader->ReadNext())
{
double avg = spReader->GetDouble(L"AVG_ID");
}
The goal of this RFC is to add overloaded methods to the various FDO reader interfaces that will accept an integer argument representing the indexed location of the property in the reader's in-memory representation of the feature or table. Readers will return the total count of properties being returned and be able to indicate which property or column is indexed a specified location. It is expected that this will speed up processing and provide a better end-user experience. Users will be able to access their data using an index into the feature reader collection as follows:
FdoPtr<FdoISelect> spSelectCmd = static_cast<FdoISelect>(mConnection->CreateCommand(FdoCommandType_Select));
spSelectCmd->SetFeatureClassName (L"Foo");
FdoPtr<FdoIdentifierCollection> spIds = spSelectCmd->GetPropertyNames ();
FdoPtr<FdoIdentifier> spId = FdoComputedIdentifier::Create(L"AVG_ID", FdoPtr<FdoExpression>(FdoExpression::Parse(L"Avg(ID)")));
spIds->Add(spId);
FdoPtr<FdoIFeatureReader> spReader = spSelectCmd->Execute ();
while (spReader->ReadNext())
{
double avg = spReader->GetDouble(0); // 0 is the index of the AVG_ID property
}
Requirements
The following FDO reader interfaces will be updated to allow for indexed access. Providers will be required to implement these methods.
Interface IReader
/// \brief
/// The FdoIReader interface provides a forward-only, read-only iterator
/// for reading feature data. FdoIReader is the base interface for
/// feature and data readers that are returned from the Select,
/// and SelectAggregates commands. FdoIReader provides the funtion
/// definitions for the GetXxxx properties and the definition of IsNull()
/// and ReadNext(). Because the initial position of the reader is prior
/// to the first item, you must call ReadNext to begin accessing any data.
class FdoIReader: public FdoIDisposable
{
public:
...
...
...
/// \brief
/// Gets the name of the property at the given ordinal position.
///
/// \param index
/// Input the position of the property.
///
/// \return
/// Returns the property name
///
FDO_API virtual FdoString* GetPropertyName(FdoInt32 index) = 0;
/// \brief
/// Gets the index of the property with the specified name.
///
/// \param propertyName
/// Input the property name.
///
/// \return
/// Returns the property index
///
FDO_API virtual FdoInt32 GetPropertyIndex(FdoString* propertyName) = 0;
/// \brief
/// Returns true if the value of the property at the specified
/// index is null.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns true if the value is null.
///
FDO_API virtual FdoBoolean IsNull(FdoInt32 index) = 0;
/// \brief
/// Gets the Boolean value of the property specified at the index position.
/// No conversion is performed, thus the property must be FdoDataType_Boolean
/// or an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the Boolean value.
///
FDO_API virtual FdoBoolean GetBoolean(FdoInt32 index) = 0;
/// \brief
/// Gets the Byte value of the property specified at the index position.
/// No conversion is performed, thus the property must be FdoDataType_Byte
/// or an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the byte value.
///
FDO_API virtual FdoByte GetByte(FdoInt32 index) = 0;
/// \brief
/// Gets the date and time value of the of the property specified at
/// the index position. No conversion is performed, thus the property
/// must be FdoDataType_DateTime or an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the date and time value.
///
FDO_API virtual FdoDateTime GetDateTime(FdoInt32 index);
/// \brief
/// Gets the double-precision floating point value of the property specified at
/// the index position. No conversion is performed, thus the property must be
/// FdoDataType_Double or an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the double floating point value
///
FDO_API virtual FdoDouble GetDouble(FdoInt32 index) = 0;
/// \brief
/// Gets the 16-bit integer value of the property specified at
/// the index position. No conversion is performed, thus the
/// property must be FdoDataType_Int16 or an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the FdoInt16 value.
///
FDO_API virtual FdoInt16 GetInt16(FdoInt32 index) = 0;
/// \brief
/// Gets the 32-bit integer value of the property specified at
/// the index position. No conversion is performed, thus the
/// property must be FdoDataType_Int32 or an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the FdoInt32 value
///
FDO_API virtual FdoInt32 GetInt32(FdoInt32 index) = 0;
/// \brief
/// Gets the 64-bit integer value of the property specified at
/// the index position. No conversion is performed, thus the
/// property must be FdoDataType_Int64 or an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the FdoInt64 value.
///
FDO_API virtual FdoInt64 GetInt64(FdoInt32 index) = 0;
/// \brief
/// Gets the Single floating point value of the property specified at
/// the index position. No conversion is performed, thus the property
/// must be FdoDataType_Single or an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the single value
///
FDO_API virtual FdoFloat GetSingle(FdoInt32 index) = 0;
/// \brief
/// Gets the string value of the property specified at the index
/// position. No conversion is performed, thus the property must
/// be FdoDataType_String or an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the string value
///
FDO_API virtual FdoString* GetString(FdoInt32 index) = 0;
/// \brief
/// Gets a LOBValue reference to the property specified at the index
/// position. The LOB is fully read in and data available.
/// Because no conversion is performed, the property must be
/// FdoDataType_BLOB or FdoDataType_CLOB etc. (a LOB type)
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the reference to LOBValue
///
FDO_API virtual FdoLOBValue* GetLOB(FdoInt32 index) = 0;
/// \brief
/// Gets a reference to the specified LOB property, specified at the index
/// position. The reference is returned as an FdoBLOBStreamReader or an
/// FdoCLOBStreamReader, to allow reading in blocks of data. Because
/// no conversion is performed, the property must be FdoDataType_BLOB
/// or FdoDataType_CLOB etc. (a LOB type) Cast the FdoIStreamReader
/// to the appropiate LOB Stream Reader.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns a reference to a LOB stream reader
///
FDO_API virtual FdoIStreamReader* GetLOBStreamReader(FdoInt32 index) = 0;
/// \brief
/// Gets the geometry value of the property, at the specified index, as
/// a byte array in FGF format. Because no conversion is performed, the
/// property must be of Geometric type; otherwise, an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the byte array in FGF format.
///
FDO_API virtual FdoByteArray* GetGeometry(FdoInt32 index) = 0;
/// \brief
/// Gets the raster object of the property at the specified index.
/// Because no conversion is performed, the property must be
/// of Raster type; otherwise, an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the raster object.
///
FDO_API virtual FdoIRaster* GetRaster(FdoInt32 index) = 0;
};
Interface IFeatureReader
/// \brief
/// The FdoIFeatureReader interface provides a forward-only, read-only iterator
/// for reading feature data. A reference to an FdoIFeatureReader is returned
/// from the Select command. Because the initial position of the FdoIFeatureReader
/// is prior to the first item, you must call ReadNext to begin accessing any data.
class FdoIFeatureReader: public FdoIReader
{
public:
...
...
...
/// \brief
/// Gets the geometry value of the property, at the specified index,
/// as a byte array in FGF format. Because no conversion is performed,
/// the property must be of Geometric type; otherwise, an exception is thrown.
/// This method is a language-specific performance optimization that returns a
/// pointer to the array data, rather than to an object that encapsulates
/// the array. The array's memory area is only guaranteed to be valid
/// until a call to ReadNext() or Close(), or the disposal of this reader
/// object.
///
/// \param index
/// Input the index of the property.
/// \param count
/// Output the number of bytes in the array.
///
/// \return
/// Returns a pointer to the byte array in FGF format.
///
FDO_API virtual const FdoByte * GetGeometry(FdoInt32 index, FdoInt32* count) = 0;
/// \brief
/// Gets the geometry value of the specified property, at the specified index,
/// as a byte array in FGF format. Because no conversion is performed, the property
/// must be of Geometric type; otherwise, an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the byte array in FGF format.
///
FDO_API virtual FdoByteArray* GetGeometry(FdoInt32 index) = 0;
/// \brief
/// Gets a reference to an FdoIFeatureReader to read the data contained in
/// the object or object collection property defined at the specified index
/// position. If the property is not an object property, an exception is thrown.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the nested feature reader
///
FDO_API virtual FdoIFeatureReader* GetFeatureObject(FdoInt32 index) = 0;
};
Interface IDataReader
/// \brief
/// The FdoIDataReader interface provides a forward-only, read-only
/// iterator for reading relational table data. A reference to an
/// FdoIDataReader is returned from the SQLCommands ExecuteReader method.
/// The initial position of the FdoIDataReader interface is prior to the first item.
/// Thus, you must call ReadNext to begin accessing any data.
class FdoIDataReader: public FdoIReader
{
public:
...
...
...
/// \brief
/// Gets the data type of the property at the specified index position.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the type of the property.
///
FDO_API virtual FdoDataType GetDataType(FdoInt32 index) = 0;
/// \brief
/// Gets the FDO property type of the property at the given index. This is used
/// to indicate if a given property is a geometric property or a data property.
/// If the property is a FdoPropertyType_DataProperty, then GetDataType
/// can be used to to find the data type of the property.
///
/// \param index
/// Input the index of the property.
///
/// \return
/// Returns the FDO property type.
///
FDO_API virtual FdoPropertyType GetPropertyType(FdoInt32 index) = 0;
};
Interface ISQLDataReader
/// \brief
/// The FdoISQLDataReader interface provides a forward-only, read-only
/// iterator for reading relational table data. A reference to an
/// FdoISQLDataReader is returned from the SQLCommands ExecuteReader method.
/// The initial position of the FdoISQLDataReader interface is prior to the first item.
/// Thus, you must call ReadNext to begin accessing any data.
class FdoISQLDataReader: public FdoIDisposable
{
public:
...
...
...
/// \brief
/// Gets the index of the column with the given column name.
///
/// \param columnName
/// Input the column name.
///
/// \return
/// Returns the column index
///
FDO_API virtual FdoInt32 GetColumnIndex(FdoString* columnName) = 0;
/// \brief
/// Returns true if the value of the specified column is null.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns true if the value is null.
///
FDO_API virtual FdoBoolean IsNull(FdoInt32 index) = 0;
/// \brief
/// Gets the data type of the column at the specified index.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the type of the column.
///
FDO_API virtual FdoDataType GetColumnType(FdoInt32 index) = 0;
/// \brief
/// Gets the FDO property type of the column at the specified index. This is used
/// to indicate if a given column is a geometric property or a data property. If the column is
/// a FdoPropertyType_DataProperty, then GetColumnType can be used to find the data type of the column.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the FDO property type of the column.
///
FDO_API virtual FdoPropertyType GetPropertyType(FdoInt32 index) = 0;
/// \brief
/// Gets the Boolean value of the specified column. No conversion is
/// performed, thus the column must be FdoDataType_Boolean or an
/// exception is thrown.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the Boolean value
///
FDO_API virtual FdoBoolean GetBoolean(FdoInt32 index) = 0;
/// \brief
/// Gets the byte value of the specified column. No conversion is
/// performed, thus the column must be FdoDataType_Byte or an
/// exception is thrown.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the byte value.
///
FDO_API virtual FdoByte GetByte(FdoInt32 index) = 0;
/// \brief
/// Gets the date time value of the specified column. No conversion
/// is performed, thus the column must be FdoDataType_DateTime or
/// an exception is thrown.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the date and time value.
///
FDO_API virtual FdoDateTime GetDateTime(FdoInt32 index) = 0;
/// \brief
/// Gets the double-precision floating point value of the specified column.
/// No conversion is performed, thus the column must be of type
/// Double or an exception is thrown.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the double value.
///
FDO_API virtual FdoDouble GetDouble(FdoInt32 index) = 0;
/// \brief
/// Gets the signed 16-bit integer value of the specified column. No conversion is
/// performed, thus the column must be FdoDataType_Int16 or an
/// exception is thrown.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the FdoInt16 value.
///
FDO_API virtual FdoInt16 GetInt16(FdoInt32 index) = 0;
/// \brief
/// Gets the signed 32-bit integer value of the specified column. No conversion is
/// performed, thus the column must be FdoDataType_Int32 or an
/// exception is thrown.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the FdoInt32 value.
///
FDO_API virtual FdoInt32 GetInt32(FdoInt32 index) = 0;
/// \brief
/// Gets the signed 64-bit integer value of the specified column. No conversion
/// is performed, thus the column must be FdoDataType_Int64 or an
/// exception is thrown.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the FdoInt64 value.
///
FDO_API virtual FdoInt64 GetInt64(FdoInt32 index) = 0;
/// \brief
/// Gets the single-precision floating point value of the specified column.
/// No conversion is performed, thus the column must be FdoDataType_Single
/// or an exception is thrown.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the single value
///
FDO_API virtual FdoFloat GetSingle(FdoInt32 index) = 0;
/// \brief
/// Gets the string value of the specified column. No conversion is
/// performed, thus the column must be FdoDataType_String or an
/// exception is thrown.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the string value.
///
FDO_API virtual FdoString* GetString(FdoInt32 index) = 0;
/// \brief
/// Gets a LOBValue reference. The LOB is fully read in and data available.
/// Because no conversion is performed, the property must be FdoDataType_BLOB or
/// FdoDataType_CLOB etc. (a LOB type)
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the reference to LOBValue
///
FDO_API virtual FdoLOBValue* GetLOB(FdoInt32 index) = 0;
/// \brief
/// Gets a reference of the specified LOB property as a FdoBLOBStreamReader or
/// FdoCLOBStreamReader etc. to allow reading in blocks of data. Because
/// no conversion is performed, the property must be FdoDataType_BLOB
/// or FdoDataType_CLOB etc. (a LOB type) Cast the FdoIStreamReader
/// to the appropiate LOB Stream Reader.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns a reference to a LOB stream reader
///
FDO_API virtual FdoIStreamReader* GetLOBStreamReader(FdoInt32 index) = 0;
/// \brief
/// Gets the geometry value of the specified column as a byte array
/// in FGF format. No conversion is performed, thus the column
/// must be of Geometric type or an exception is thrown.
///
/// \param index
/// Input the position of the column.
///
/// \return
/// Returns the FGF byte array value.
///
FDO_API virtual FdoByteArray* GetGeometry(FdoInt32 index) = 0;
};
Default implementation
A high level class diagram of existing reader related interfaces in FDO is as illustrated below.
Both FdoISqlReader and FdoIDataReader have the function to get the column name or property name according to the index. With this capability, it should be pretty easy to provide default implementation for all access by index functions. Take GetInt32() as an example:
FdoInt32 GetInt32(FdoInt32 index)
{
FdoStringP propName = GetPropertyName(index);
return GetInt32(propName);
}
As for FdoIFeatureReader, there is no such function of FdoString* GetPropertyName(FdoInt32 index). The default implementation will use GetClassDefinition() to enumerate the property names. The index order is the same as the collection of properties returned by the class definition.
Managed FDO API
The FDO Managed Interfaces will be updated in a similar manner to reflect the proposed changes.
Provider Implementation
Providers will be required to implement the above functionality. Instead of each provider to implement all of newly added pure virtual functions, three utility abstract classes are added to provide the default implementation as mentioned above under Fdo/Utilities/Common: FdoDefaultFeatureReader, FdoDefaultDataReader, FdoDefaultSqlReader. Then the provider specific implementation will simply derive from the new classes. That involves changing one line in header file for each provider (some providers may not link FdoCommon and need to add that dependancy). Take FdoDefaultDataReader as an example:
class FdoDefaultDataReader: public FdoIDataReader
{
public:
virtual bool FdoIReader::GetBoolean(FdoString* propertyName) = 0;
virtual FdoByte FdoIReader::GetByte(FdoString* propertyName) = 0;
virtual FdoDateTime FdoIReader::GetDateTime(FdoString* propertyName) = 0;
virtual double FdoIReader::GetDouble(FdoString* propertyName) = 0;
virtual FdoInt16 FdoIReader::GetInt16(FdoString* propertyName) = 0;
virtual FdoInt32 FdoIReader::GetInt32(FdoString* propertyName) = 0;
virtual FdoInt64 FdoIReader::GetInt64(FdoString* propertyName) = 0;
virtual float FdoIReader::GetSingle(FdoString* propertyName) = 0;
virtual FdoString* FdoIReader::GetString(FdoString* propertyName) = 0;
virtual FdoLOBValue* FdoIReader::GetLOB(FdoString* propertyName) = 0;
virtual FdoIStreamReader* FdoIReader::GetLOBStreamReader(const wchar_t* propertyName ) = 0;
virtual bool FdoIReader::IsNull(FdoString* propertyName) = 0;
virtual FdoByteArray* FdoIReader::GetGeometry(FdoString* propertyName) = 0;
virtual FdoIRaster* FdoIReader::GetRaster(FdoString* propertyName) = 0;
virtual FdoDataType FdoIDataReader::GetDataType(FdoString* propertyName) = 0;
virtual FdoPropertyType FdoIDataReader::GetPropertyType(FdoString* propertyName) = 0;
virtual FdoBoolean IsNull(FdoInt32 index);
virtual FdoBoolean GetBoolean(FdoInt32 index);
virtual FdoByte GetByte(FdoInt32 index);
virtual FdoDateTime GetDateTime(FdoInt32 index);
virtual FdoDouble GetDouble(FdoInt32 index);
virtual FdoInt16 GetInt16(FdoInt32 index);
virtual FdoInt32 GetInt32(FdoInt32 index);
virtual FdoInt64 GetInt64(FdoInt32 index);
virtual FdoFloat GetSingle(FdoInt32 index);
virtual FdoString* GetString(FdoInt32 index);
virtual FdoLOBValue* GetLOB(FdoInt32 index);
virtual FdoIStreamReader* GetLOBStreamReader(FdoInt32 index);
virtual FdoByteArray* GetGeometry(FdoInt32 index);
virtual FdoIRaster* GetRaster(FdoInt32 index);
virtual FdoString* GetPropertyName(FdoInt32 index);
virtual const FdoByte * GetGeometry(FdoInt32 index, FdoInt32* count);
virtual FdoByteArray* GetGeometry(FdoInt32* index);
};
Note: it needs to expose access by property name functions as well, otherwise they will be hiden by access by index functions that can't be visible to derived classes. All pure virtual functions for access by index are overrided with the default implementation in a similar way. e.g.
FdoBoolean FdoDefaultDataReader::IsNull(FdoInt32 index)
{
FdoStringP propName = GetPropertyName(index);
Return IsNull(propName);
}
Changes to provider specific implementation:
Class XYZDataReader: public FdoDefaultDataReader
{
// no other changes
}
It is expected that resourcing will be supplied to modify all the existing FDO Open Source providers. Unless there are noticeable performance complaints, all of the open source providers will use the default implementation for this time.
Test Plan
Existing FDO Core and Provider level unit tests will be expanded to test the proposed enhancements defined above.
Funding/Resources
Autodesk to provide resources / funding
Addendum, Sept. 15, 2009
Previously, there is not a function to get the property index given the name of the property like FdoInt32 GetPropertyIndex(FdoString* propertyName); To use access by index functions on FdoIFeatureReader and FdoIDataReader will be hard for API users as they need to know beforehand how index is mapping to property name and mapping is different for different provider implementations like SqlLite/SDF and RDBMS. E.g. with the same FDO schema definition, SqlLite and RDBMS will arrange the property order returned from GetClassDefinition() differently:
- If you don’t set selected property names be returned from an FdoISelect command
SqlLite provider will arrange the properties in the order: Identity property -> Geometry property -> the left will be arranged in the order of how they are added into the class definition, like ID -> geometry -> datetime-> double For RDBMS provider the order is: identity property -> the left is ordered by the first character of the property name like ID -> datetime-> double -> geometry
- If you do set selected property names to be returned from an FdoISelect command
SqlLite provider will respect the order of how they are added to selected properties. SDF provider will respect the order of corresponding properties defined in the original class definition(without setting selected properties)
So we added another function of FdoInt32 GetPropertyIndex(FdoString* propertyName) in this RFC in order for API users to use the access by index on all readers conveniently and correctly. Now both of the property name and index related functions are moved to FdoIReader as they are needed for both FdoIFeatureReader and FdoIDataReader.
FdoString* GetPropertyName(FdoInt32 index); FdoInt32 GetPropertyIndex(FdoString* propertyName);
To make sure that the access by index is easy to use by client applications, this RFC is including an additional requirement on the indexing.
The index order supported by the feature reader must be the same as the order of the properties described by the class definition that is returned by the feature reader. This allows application developers to have a predictable ordering and not have to maintain additional mapping tables at their end.
The default feature reader implementation described above will follow this requirement. Providers that do not implement these new functions natively will not have to change for applications to be able to use the index access. For providers that implement these new functions natively, this index ordering requirement must be used.
