FDO RFC 55 - Convert FdoDecimal(p,0) datatype to FdoIntXX for SHP Provider

This page contains an change request (RFC) for the FDO Open Source project. More FDO RFCs can be found on the RFCs page.

Status

RFC Template Version	(1.0)
Submission Date	June 6, 2011
Last Modified	Dan Stoica January 14, 2011
Author	Dan Stoica
RFC Status	For review
Implementation Status	Prototype
Proposed Milestone	3.6.0.0
Assigned PSC guide(s)	Greg Boone
Voting History
+1
+0
-0
-1

Overview

This RFC is for handling the cases where DBF files contain DECIMAL(precision, 0) columns.

Motivation

The motivation is well explained by: Ticket #365 [SHP Provider]: When executing IApplySchema, Int32 properties are converted to Decimal properties ​http://trac.osgeo.org/fdo/ticket/365

If you execute an IApplySchema on a SHP connection. Any Int32 properties in any class definitions inside the schema (to be applied) will be converted to Decimal properties once the schema has been applied. This does not happen if the Int32 property happens to be an Identity property. <<

For further clarification "will be converted to Decimal properties" refer to FdoDecimal properties.

Note storing !FdoInt32 as Decimals is expected since by specification DBF files natively do not have an integer type, but only DECIMAL(precision, scale) datatype. The same apply to double and single precision datatypes. Hence a integer type like !FdoInt32 is mapped to a DECIMAL(11, 0) column type.

The impact is on Describe Schema when currently a FdoDecimal is returned for a DECIMAL column even in the cases when the scale is zero and most probably the original type was a !FdoInt32.

This RFC is required since reading back a !FdoInt32 instead of a FdoDecimal is a change in behavior. For completitude !FdoInt16 and !FdoInt64 have been added to the scope.

Proposed Solution

Modify the FDO SHP provider:

• the DBF to FDO datatype mapping (physical to logical conversion) needs a small refinement. In the case of DECIMAL(precision, scale) check the scale value. When the scale is 0 (zero) and depending on the precision return !FdoInt16, !FdoInt32 or !FdoInt64 rather than an FdoDecimal. Note SHP provider currently does not support !FdoInt64 or !FdoInt16 datatypes and they have to be added to the schema capabilities.

Below the mapping between FdoIntXX datatypes and Decimal precision:

Fdo type	precision	range
!FdoInt16	1...6	-32,768 to 32,767
!FdoInt32	7...11	-2,147,483,648 to 2,147,483,647
!FdoInt64	12...20	-9,223,372,036,854,775,808 to 9,223,372,036,854,775,807

Note: beyond the !FdoInt64 range the current behaviour will be preserved, i.e. the property type will remain FdoDecimal.

But assume a SHP file with a column of DECIMAL(4,0). DescribeSchema will return that as a FdoInt16. On an insert operation we need to make sure the user has enough information to avoid values out of (-999, 999) range. Or in an ApplySchema operation we need to treat the Int16 as DECIMAL(6,0) as opposed to DECIMAL(4,0) to be sure it can handle the full range. Therefore a suplimentary change is needed: the FdoPropertyDefinition will return FdoIntXX datatype but retain the original Precision (the Scale is implicit zero).

Implications

• In the case of physical DECIMAL(precision, scale) column the corresponding logical FDO property type will be set according to the scale. The caller must expect either a FdoDecimal or a FdoIntXX when reading a SHP schema.

• This is a behavior change for the SHP provider. Before this change, SHP would return only FdoDecimal for all numerical FDO types. It would never return an FdoIntXX except for the identity property (which actually is not stored).

• The provider will return a FdoIntXX even for properties which originally were DECIMAL(p, 0). This issue can be mitigated by noting the SHP provider already does silent corrections on the fly in certain cases, like truncating the property names when found too long (both on reading and writing).

• The FdoDataPropertyDefinition documentation will be change to reflect that precision apply to all integer types not only for decimals. In case the precision is not set (zero) then the full range is assumed. This is the current behavior.

    /// \brief
    /// Gets the precision (total number of digits) of a decimal or integer data property.
    /// This value does not apply to any other FdoDataType.
    /// 
    FDO_API FdoInt32 GetPrecision();

Test Plan

• Enhance the unit tests and add roundtripping fidelity tests for !FdoInt16, !FdoInt32 and !FdoInt64 properties.
• Add tests to check for out of range values. Make sure the out-of-range values on insert are detected.

Funding/Resources

Autodesk to provide resources/funding.