numeric field overflow

[strk]

Importing a shapefile to PostGIS gets required numeric field precision wrong:

...
ALTER TABLE "nycg" ADD COLUMN "shape_area" NUMERIC(19,11)
...
ERROR:  numeric field overflow
DETAIL:  A field with precision 19, scale 11 must round to an absolute value less than 10^8

The value being inserted is:

1830021049.74000000954

It could be a bug in how the value is printed (wrong printf format). Indeed libreoffice interprets the number as

1830021049.74000000000

Using layer creation option PRECISION=NO works around this (in GDAL 1.10.1) using a float8 as the result. Also in this case the number passed to INSERT or COPY have the extra precision, but PostgreSQL is tolerant about that and simply discards the extra precision, giving a round 1830021049.74 as output.

[Even Rouault]

Looks like the inconsistency is in the dbf associated with the shapefile. Indeed with N(19,11) formatting, the integer part of the number should have a maximum of 19-11-1=7 digits (consistant with the < 10^{8 PG error message). And here it has 10 digits.}

As far as the trailing extra precision digits, this is simply due to printf rounding errors :

>>> '%19.11f' % 1830021049.74
'1830021049.74000000954'

Furhtermore as GDAL doesn't have a real numeric datatype but uses double precision numbers internally, even a valid N(19,11) number couldn't be represented exactly :

>>> '%19.11f' % 1234567.12345678901
'1234567.12345678895'

[strk]

Thanks for looking at this. I wonder if GDAL could notice the malformed numbers earlier in the process to provide a more detailed error message, like "dbf is malformed due to this and that, try with PRECISION=no" ?

[Jukka Rahkonen]

If this kind of malformed dbf files are not very common (like created with all certain versions of ESRI products or GDAL) I think that adding dbf sanitazer features into GDAL is not worth the pain. Reopen if you don't agree.