Opened 14 years ago

Closed 10 years ago

#1488 closed defect (duplicate)

[PATCH] real-handling in .DBF-files

Reported by: jesper@… Owned by: warmerdam
Priority: normal Milestone:
Component: OGR_SF Version: 1.4.0
Severity: major Keywords: shape driver

Description (last modified by Mateusz Łoskot)

Hi There

Think I found a bug in the ogr2ogr tool (from inside the windows FWTools1.2.0):

when I try to convert an area that has large real-numbers, the sign is not treated as I expect. After convertion from source to targetfile (with only the affected area inside) the atribute-number 57520907013796 is ,. Same thing with FEATAREA. See the headers below (from ogrinfo):

Sourcefile (largebug_region.shp):
ID: Real (15.0)
FEATTYP: Integer (4.0)
FEATAREA: Real (15.0)
FEATPERIM: Real (15.0)
POSTCODE: String (10.0)
  ID (Real) =  57520907013796
  FEATTYP (Integer) = 3136
  FEATAREA (Real) =     11255420040
  FEATPERIM (Real) =         1202210
  POSTCODE (String) = 930

but in the target (using ogr2ogr largebug_target.shp largebug_region.shp) data looks like this

ID: Real (15.0)
FEATTYP: Integer (4.0)
FEATAREA: Real (15.0)
FEATPERIM: Real (15.0)
POSTCODE: String (10.0)
  ID (Real) =     -1589981532
  FEATTYP (Integer) = 3136
  FEATAREA (Real) =     -1629481848
  FEATPERIM (Real) =         1202210
  POSTCODE (String) = 930

Thought you wanted to know. Let me know if you need the test-file.

regards Jesper

Attachments (2) (62.0 KB) - added by jesper@… 14 years ago.
the .shp and dbf test files
gdal_svn_trunk_dbfopen_fix_1488.patch (9.8 KB) - added by Even Rouault 13 years ago.

Download all attachments as: .zip

Change History (7)

Changed 14 years ago by jesper@…

Attachment: added

the .shp and dbf test files

comment:3 Changed 14 years ago by Mateusz Łoskot

Description: modified (diff)

comment:4 Changed 13 years ago by Even Rouault

Keywords: shape driver added
Milestone: 1.6.0
Summary: real-handling in .DBF-files[PATCH] real-handling in .DBF-files

I confirm this bug exists. It's due to the following code snippet in dbfopen.c :

	if( psDBF->panFieldDecimals[iField] == 0 )
            int		nWidth = psDBF->panFieldSize[iField];

            if( (int) sizeof(szSField)-2 < nWidth )
                nWidth = sizeof(szSField)-2;

	    sprintf( szFormat, "%%%dd", nWidth );
	    sprintf(szSField, szFormat, (int) *((double *) pValue) );
	    if( (int)strlen(szSField) > psDBF->panFieldSize[iField] )
	        szSField[psDBF->panFieldSize[iField]] = '\0';
                nRetResult = FALSE;

	    strncpy((char *) (pabyRec+psDBF->panFieldOffset[iField]),
		    szSField, strlen(szSField) );

Here, we assume that if psDBF->panFieldDecimals[iField] == 0 we can safely cast to an int, which is wrong.

To fix that and add enhanced security to those casts, I propose to add an extra argument to DBFWriteAttribute that gives the type of the passed data pointer. The patch also includes use of psDBF->sHooks.Error instead of CPLError / fprintf.

Changed 13 years ago by Even Rouault

comment:5 Changed 12 years ago by szigeti


I've had the same problem with the attributes of type Real(15,0). It is due to the limited value repreentation ability of the system -- type int (sizeof(int) is usually 4) cannot store a number that is 15 digits long. So casting the *pValue to (int) spoils the data. Using type long long is a better solution thus the following patch of dbfopen.c fixed my problem (version 1.5.2):

< 	    sprintf( szFormat, "%%%dd", nWidth );
< 	    sprintf(szSField, szFormat, (int) *((double *) pValue) );
> 	    sprintf( szFormat, "%%%dlld", nWidth );
> 	    sprintf(szSField, szFormat, (long long) *((double *) pValue) );

Note that this is not a general solution since long long (assuming that sizeof(long long equals 8)) provides only a 64bit representation (~19digits).

I would suggest creating a general type/class for numbers (even for the long ones which do not fit into type double or long long) and use this type internally or deal with the numbers as character-strings..

it's up to you:)

comment:6 Changed 12 years ago by warmerdam

Milestone: 1.6.0
Priority: highestnormal

I'm not sure how this got set to highest priority. Resetting to normal.

I have avoided incorporating a 64bit integer type into shapelib due to the complication it causes in the software configuration - so far quite simple for Shapelib.

I do not forsee this being dealt with in 1.6.0.

comment:7 Changed 10 years ago by warmerdam

Resolution: duplicate
Status: newclosed

I'm making #3615 the official ticket to address wide integer fields in dbf.

Note: See TracTickets for help on using tickets.