Opened 10 years ago

Closed 17 months ago

#792 closed defect (wontfix)

dbf driver: wrong column data type

Reported by: martinl Owned by: grass-dev@…
Priority: normal Milestone: 7.0.7
Component: Database Version: svn-trunk
Keywords: dbf, data type Cc:
CPU: Unspecified Platform: Unspecified

Description

DBF driver in GRASS 7 reports 'double' even if decimals is zero, e.g.

 D2/5:   DBFFieldType 2
D3/5: add_column(): tab = 2, type = 3, name = cat, width = 11, decimals = 0
DOUBLE PRECISION|cat

Attached patch forces 'integer' if 'double' with decimal = 0 is detected. I am not sure if it's right approach.

Attachments (1)

dbf-int.diff (559 bytes) - added by martinl 10 years ago.
dbf data type

Download all attachments as: .zip

Change History (16)

Changed 10 years ago by martinl

Attachment: dbf-int.diff added

dbf data type

comment:1 Changed 10 years ago by mmetz

I would first like to know why in grass7 integer fields with width > 10 are considered double, contrary to grass6.x.

See also gdal ticket 809.

In particular, why does the Makefile for the dbf driver now include $(GDALLIBS) but not in grass6.x?

Markus M

comment:2 in reply to:  1 Changed 10 years ago by martinl

Replying to mmetz:

I would first like to know why in grass7 integer fields with width > 10 are considered double, contrary to grass6.x.

See also gdal ticket 809.

Right, same problem with 'OGR' driver. There is no info about decimals [1].The attached patch is ugly workaround which cannot be applied in OGR driver. Martin

[1] http://www.gdal.org/ogr/classOGRFieldDefn.html

comment:3 in reply to:  1 ; Changed 10 years ago by martinl

Replying to mmetz:

In particular, why does the Makefile for the dbf driver now include $(GDALLIBS) but not in grass6.x?

In the result dbf driver uses GDAL not ShapeLib?. Removing $(GDALLIBS) from Makefile would be a quick solution. Anyway there is still problem with OGR driver. For every OGR layers is key column (if defined) reported as 'double' which breaks most of GRASS modules.

comment:4 in reply to:  3 ; Changed 10 years ago by mmetz

Replying to martinl:

Replying to mmetz:

In particular, why does the Makefile for the dbf driver now include $(GDALLIBS) but not in grass6.x?

In the result dbf driver uses GDAL not ShapeLib?. Removing $(GDALLIBS) from Makefile would be a quick solution.

Sure, but I guess there is a reason for the new presence of $(GDALLIBS) for the dbf driver? It seems it was added by Glynn in r38906. But it's working fine after I removed $(GDALLIBS) from the Makefile.

comment:5 Changed 10 years ago by hamish

could you post here which DBF from the nc_ dataset you are using for tests?

tx

comment:6 in reply to:  5 ; Changed 10 years ago by hamish

Replying to hamish:

could you post here which DBF from the nc_ dataset you are using for tests?

fyi,

$ dbview -e grassdata/nc_spm_08/PERMANENT/dbf/geology.dbf  | head

Field Name      Type    Length  Decimal Pos
cat               N        11       0
onemap pro        N        20       6
Perimeter         N        20       6
Geol250           N        11       0
Geol250 id        N        11       0
Geo name          C         6       0
Shape area        N        20       6
Shape len         N        20       6

comment:7 in reply to:  5 Changed 10 years ago by martinl

Replying to hamish:

could you post here which DBF from the nc_ dataset you are using for tests?

Sorry, I am not getting the point. AFAIU problem is a bug in GDAL/OGR when as a result 'integer' is detected as 'double precision', the very same for OGR driver in GRASS of course.

Martin

comment:8 in reply to:  6 Changed 10 years ago by martinl

Replying to hamish:

Replying to hamish:

could you post here which DBF from the nc_ dataset you are using for tests?

> $ dbview -e grassdata/nc_spm_08/PERMANENT/dbf/geology.dbf  | head
> 
> Field Name      Type    Length  Decimal Pos
> cat               N        11       0
> onemap pro        N        20       6
> Perimeter         N        20       6
> Geol250           N        11       0
> Geol250 id        N        11       0
> Geo name          C         6       0
> Shape area        N        20       6
> Shape len         N        20       6

Problem are not the data, but the bug in OGR. If you use OGR for reading dbf, 'cat' is detected as 'double precision', same for PostGIS layer connected by OGR driver, etc.

comment:9 in reply to:  4 ; Changed 10 years ago by glynn

Replying to mmetz:

In particular, why does the Makefile for the dbf driver now include $(GDALLIBS) but not in grass6.x?

Sure, but I guess there is a reason for the new presence of $(GDALLIBS) for the dbf driver? It seems it was added by Glynn in r38906. But it's working fine after I removed $(GDALLIBS) from the Makefile.

r38906 was a scripted change which matched each program with the libraries which exported the symbols which were imported by the program. In the case of the DBF driver, the program imports the various DBF* symbols, GDAL exports these symbols, therefore $(GDALLIBS) was added to the Makefile.

The process didn't handle the case where the imported symbols are exported by multiple libraries (in this case, GDAL and ShapeLib). AFAICT, the DBF driver is the only program which uses ShapeLib.

comment:10 in reply to:  9 ; Changed 10 years ago by mmetz

Replying to glynn:

Replying to mmetz:

In particular, why does the Makefile for the dbf driver now include $(GDALLIBS) but not in grass6.x?

Sure, but I guess there is a reason for the new presence of $(GDALLIBS) for the dbf driver? It seems it was added by Glynn in r38906. But it's working fine after I removed $(GDALLIBS) from the Makefile.

r38906 was a scripted change which matched each program with the libraries which exported the symbols which were imported by the program. In the case of the DBF driver, the program imports the various DBF* symbols, GDAL exports these symbols, therefore $(GDALLIBS) was added to the Makefile.

The process didn't handle the case where the imported symbols are exported by multiple libraries (in this case, GDAL and ShapeLib). AFAICT, the DBF driver is the only program which uses ShapeLib.

Ah, ok. Reading around a bit (documentation and previous discussions), it seems that the dbf driver is supposed to use !Shapelib instead of GDAL. I removed $(GDALLIBS) from the Makefile in r39594 (it's not the first time that is done), works for me, please test.

Markus M

comment:11 in reply to:  10 Changed 6 years ago by neteler

Keywords: dbf added

Replying to mmetz:

Replying to glynn:

Replying to mmetz:

...

The process didn't handle the case where the imported symbols are exported by multiple libraries (in this case, GDAL and ShapeLib). AFAICT, the DBF driver is the only program which uses ShapeLib.

Ah, ok. Reading around a bit (documentation and previous discussions), it seems that the dbf driver is supposed to use !Shapelib instead of GDAL. I removed $(GDALLIBS) from the Makefile in r39594 (it's not the first time that is done), works for me, please test.

Is there a reason to keep the private (older) shapelib copy instead of using GDAL?

See also https://bugzilla.redhat.com/show_bug.cgi?id=1087604

comment:12 Changed 4 years ago by martinl

Milestone: 7.0.07.0.5

comment:13 Changed 3 years ago by neteler

Milestone: 7.0.57.0.6

comment:14 Changed 2 years ago by neteler

Milestone: 7.0.67.0.7

comment:15 Changed 17 months ago by martinl

Resolution: wontfix
Status: newclosed

No activity for long time. DBF is not recommended to use in GRASS 7. Closing. Feel free to reopen if needed.

Note: See TracTickets for help on using tickets.