Opened 14 years ago
Closed 14 years ago
#635 closed defect (fixed)
FdoDataValue: error converting max + 1
| Reported by: | brentrobinson | Owned by: | brentrobinson |
|---|---|---|---|
| Priority: | major | Milestone: | 3.6.0 |
| Component: | FDO API | Version: | 3.5.0 |
| Severity: | 2 | Keywords: | DataValue convert precision |
| Cc: | External ID: |
Description
The following:
FdoPtr<FdoDoubleValue> dblVal = FdoDoubleValue::Create(pow((double)2,(double)63));
FdoPtr<FdoInt64Value> i64Val = FdoInt64Value::Create(dbiVal, false, false, true);
will fail with "Cannot convert value 9.22337203685478e+018 to int64 without loss of precision", if run on a Windows 64bit machine.
This should have succeeded since dblVal > FdoInt64Max and conversion was attempted with truncate=true.
Note:
See TracTickets
for help on using tickets.
Revision: 5446 Author: brentrobinson Date: 2:57:08 PM, Wednesday, March 31, 2010 Message: Ticket#635: Handle floating point to integral conversion where floating point value = max + 1, where max is the maximum value for the output type.
Modified : /trunk/Fdo/Unmanaged/Inc/Fdo/Expression/DataValue.h Modified : /trunk/Fdo/Unmanaged/Src/Fdo/Expression/ByteValue.cpp Modified : /trunk/Fdo/Unmanaged/Src/Fdo/Expression/Int16Value.cpp Modified : /trunk/Fdo/Unmanaged/Src/Fdo/Expression/Int32Value.cpp Modified : /trunk/Fdo/Unmanaged/Src/Fdo/Expression/Int64Value.cpp Modified : /trunk/Fdo/Unmanaged/Src/Fdo/Expression/SingleValue.cpp