FDO RFC #30 - DataValue Type Conversion

This page contains a request for comments document (RFC) for the FDO Open Source project. More FDO RFCs can be found on the RFCs page.

Status

RFC Template Version	(1.0)
Submission Date	(Date/Time submitted)
Last Modified	(your name here) Timestamp
Author	Brent Robinson
RFC Status	draft
Implementation Status	pending
Proposed Milestone	3.5
Assigned PSC guide(s)	(when determined)
Voting History	(vote date)
+1
+0
-0
-1

Overview

The purpose of this RFC is to enhance the FDO API to support converting FDO data values between various types.

Motivation

There are a number of places in core FDO and Provider source code where data values are converted from one type to another. These data values include property values for features and property constraint values. Also, as various FDO client applications are written, some of them may also need to do data type conversion.

One current example is the FdoIDescribeSchema implementation for the RDBMS providers, which extracts check constraints from SQL expressions obtained from the RDBMS, and converts them into FDO Property Value Constraints. These constraints contain data values that represent ranges or lists of allowed values. The data types determined by the expression parser do not always exactly match the data type for the property that owns the constraint. When the types differ, the data values are converted to the property's data type.

Another example is when an SDF file has a feature class with an int64 identity property called ID, and an FdoISelect is performed with the following string filter:

ID = 5

When the filter is parsed, the value 5 becomes an int32 value. Since the filter is on the identity property, the SDF provider optimizes the select by doing an indexed lookup. However, for the lookup to work, the value must be converted to the property type (int32->int64).

Data value type conversions are not always trivial and the supporting runtime functions sometimes differ between Win32 and Linux (e.g. converting between string and int64). Implementing these conversions in different places leads to the following problems:

• there is no consistent handling of type conversions. For example, different parts of FDO might handle overflows (value is not valid for the destination type) differently

• code duplication. More expensive maintenance if the same fix has to be applied in multiple places (this can easily happen when the type conversion code is cloned to different places)

• more difficult for developers to find the type conversion functions currently available, since these can be in various places

Data values are encapsulated in the FdoDataValue class and its derivations. There is a derivation for each data type supported by FDO. Example data types include various integer types (8bit, 16bit, 32bit, 64bit), string and datetime.

The FdoDataValue class effectively provides this type conversion now, by supporting the conversion from strings to other data types and vice-versa. For example, a double can be converted to an int32 by converting it to a string and then to an int32. However, providing a more direct conversion, without converting to and from strings, would provide better performance. It also allows very large or small values to be converted more accurately.

It is also possible to convert between various numeric types through simple casting. However, this leads to source code with sizeable switch statements when converting between two arbitrary data types.

Proposed Solution

This RFC proposes that data type conversions be encapsulated in new API functions on the FdoDataValue class.

Direct conversion from a source to a destination type will be supported by adding Create functions to each destination type, which take a value of the source type as a parameter.

The following describes the changes to the unmanaged API. However, equivalent changes will also be made to the managed API.

Main Functions

The following functions will be added:

FDO_API static FdoDataValue* FdoDataValue::Create(
     FdoDataType dataType, 
     FdoDataValue* src, 
     FdoBoolean nullIfIncompatible = false,
     FdoBoolean shift = true,
     FdoBoolean truncate = false
);
 
FDO_API static FdoBooleanValue* FdoBooleanValue::Create(
     FdoDataValue* src, 
     FdoBoolean nullIfIncompatible = false,
     FdoBoolean shift = true,
     FdoBoolean truncate = false
 );
 
FDO_API static FdoByteValue* FdoByteValue::Create(
     FdoDataValue* src, 
     FdoBoolean nullIfIncompatible = false,
     FdoBoolean shift = true,
     FdoBoolean truncate = false
 );
 
FDO_API static FdoDecimalValue* FdoDecimalValue::Create(
     FdoDataValue* src, 
     FdoBoolean nullIfIncompatible = false,
     FdoBoolean shift = true,
     FdoBoolean truncate = false
 );
 
FDO_API static FdoDoubleValue* FdoDoubleValue::Create(
     FdoDataValue* src, 
     FdoBoolean nullIfIncompatible = false,
     FdoBoolean shift = true,
     FdoBoolean truncate = false
 );
 
FDO_API static FdoInt16Value* FdoInt16Value::Create(
     FdoDataValue* src, 
     FdoBoolean nullIfIncompatible = false,
     FdoBoolean shift = true,
     FdoBoolean truncate = false
 );
 
FDO_API static FdoInt32Value* FdoInt32Value::Create(
     FdoDataValue* src, 
     FdoBoolean nullIfIncompatible = false,
     FdoBoolean shift = true,
     FdoBoolean truncate = false
 );
 
FDO_API static FdoInt64Value* FdoInt64Value::Create(
     FdoDataValue* src, 
     FdoBoolean nullIfIncompatible = false,
     FdoBoolean shift = true,
     FdoBoolean truncate = false
 );
 
FDO_API static FdoSingleValue* FdoSingleValue::Create(
     FdoDataValue* src, 
     FdoBoolean nullIfIncompatible = false,
     FdoBoolean shift = true,
     FdoBoolean truncate = false
 );
FDO_API static FdoStringValue* FdoStringValue::Create(
     FdoDataValue* src, 
     FdoBoolean nullIfIncompatible = false,
     FdoBoolean shift = true,
     FdoBoolean truncate = false
 );
 
FDO_API static FdoCLOBValue* FdoCLOBValue::Create(
     FdoDataValue* src, 
     FdoBoolean nullIfIncompatible = false,
     FdoBoolean shift = true,
     FdoBoolean truncate = false
 );

where:

dataType – specifies the destination type. This parameter is only specified for FdoDataValue::Create(). For the other functions, the destination type is determined by the return type.

src – specifies the source type and value. It is converted to an FdoDataValue of the destination type.

nullIfIncompatible – determines what happens if the source and destination types are incompatible (e.g. pass FdoDateTimeValue as src to FdoBoolean::Create()):

true: returns a null FdoDataValue (FdoDataValue::IsNull() = true). Note that the pointer returned is not null, the FdoDataValue returned has its null flag set

false: throw an exception.

shift – determines what happens if conversion would cause the value to change. This can happen when converting between numeric types and the destination type does not have enough precision to hold the value as is (e.g convert 1.9 from Double to Int32 ). Applicable only when both source and destination types are one of Byte, Decimal, Double, Int16, Int32, Int64 or Single:

true: shift the value by rounding it to the precision allowed by the destination type

false: depends on the value of nullIfIncompatible:

true: return a null FdoDataValue (FdoDataValue::IsNull() = true). Note that the pointer returned is not null, the FdoDataValue returned has its null flag set

false: throw an exception

truncate – determines what happens if the value is outside the valid range for the destination type (e.g. convert 1000000 from FdoInt32Value to FdoInt16Value). Applicable only when both source and destination types are one of Boolean, Byte, Decimal, Double, Int16, Int32, Int64 or Single:

true: truncate the value to:

the maximum value for the destination type if the input value is greater than the maximum value

the minimum value for the destination type if the input value is less than the minimum value

false: depends on the value of nullIfIncompatible:

true: return a null FdoDataValue (FdoDataValue::IsNull() = true). Note that the pointer returned is not null, the FdoDataValue returned has its null flag set

false: throw an exception

When the source type is numeric and the destination type is Boolean, the behaviour differs from the above and is as follows:

true: convert 0 to false and other values to true

false: convert 0 to false and 1 to true. Throw an exception for other values

See the chart in the attached document for information on which pairs of types are compatible and which are incompatible.

Supporting Functions

The following functions will be added:

FDO_API FdoInt64 FdoStringP::ToInt64()

which converts a string to int64 value, and

FDO_API FdoStringP::FdoStringP( FdoInt64 value )

to convert an int64 to string. These functions hide the Windows and Linux specific differences in doing these conversions.

Implications

This section allows discussion of the repercussions of the change, such as whether there will be any breakage in backwards compatibility, if documentation will need to be updated, etc.

Test Plan

How the proposed change will be tested, if applicable. New unit tests should be detailed here???

Funding/Resources

This section will confirm that the proposed feature has enough support to proceed. This would typically mean that the entity making the changes would put forward the RFC, but a non-developer could act as an RFC author if they are sure they have the funding to cover the change.