Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#4163 closed enhancement (fixed)

FGDB options for controlling precision grid

Reported by: pramsey Owned by: Even Rouault
Priority: high Milestone: 1.9.0
Component: default Version: svn-trunk
Severity: normal Keywords:
Cc:

Description

The FGDB file creates features on a precision grid, so users should be able to modify the default grid when creating new layers. Attached patch allows this.

Attachments (2)

fgdb-precision.patch (18.2 KB) - added by pramsey 10 years ago.
fgdb-precision-2.patch (20.1 KB) - added by pramsey 10 years ago.
second try

Download all attachments as: .zip

Change History (10)

Changed 10 years ago by pramsey

Attachment: fgdb-precision.patch added

comment:1 Changed 10 years ago by Even Rouault

Paul, I applied the patch in my dev tree, and I get really nasty crashes when running the ogr_fgdb.py test (or when doing ogr2ogr -f filegdb some.gdb some.shp). This is with Linux 64bit.

Here are the Valgrind traces with ogr_fgdb.py :

==12046==
==12046== Invalid write of size 1
==12046==    at 0xF08F0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF065547: ShapeCompressor::CompressMultipoint(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0665EA: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F1BCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F953C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xEFFEE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0x75B6CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12046==    by 0x773FD84: OGR_L_CreateFeature (ogrlayer.cpp:442)
==12046==    by 0x15953442: _wrap_Layer_CreateFeature (ogr_wrap.cpp:3431)
==12046==    by 0x4A7A06: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A9670: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==12046==    by 0x4A7808: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==  Address 0x16d260c0 is 0 bytes after a block of size 1,024 alloc'd
==12046==    at 0x4C27878: malloc (vg_replace_malloc.c:236)
==12046==    by 0xF096F81: IOBuffer::Allocate(int, AllocationType) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0EBEEA: StandardDatafile::SetupOutputBuffers(int) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F9409: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xEFFEE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0x75B6CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12046==    by 0x773FD84: OGR_L_CreateFeature (ogrlayer.cpp:442)
==12046==    by 0x15953442: _wrap_Layer_CreateFeature (ogr_wrap.cpp:3431)
==12046==    by 0x4A7A06: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A9670: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==12046==    by 0x4A7808: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A854F: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==
==12046==
==12046== More than 10000000 total errors detected.  I'm not reporting any more.
==12046== Final error counts will be inaccurate.  Go fix your program!
==12046== Rerun with --error-limit=no to disable this cutoff.  Note
==12046== that errors may occur in your program without prior warning from
==12046== Valgrind, because errors are no longer being displayed.
==12046==
==12046==
==12046== Process terminating with default action of signal 11 (SIGSEGV)
==12046==  Access not within mapped region at address 0x17B97000
==12046==    at 0xF08F0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF065547: ShapeCompressor::CompressMultipoint(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0665EA: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F1BCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F953C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xEFFEE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0x75B6CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12046==    by 0x773FD84: OGR_L_CreateFeature (ogrlayer.cpp:442)
==12046==    by 0x15953442: _wrap_Layer_CreateFeature (ogr_wrap.cpp:3431)
==12046==    by 0x4A7A06: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A9670: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==12046==    by 0x4A7808: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==  If you believe this happened as a result of a stack
==12046==  overflow in your program's main thread (unlikely but
==12046==  possible), you can try to increase the size of the
==12046==  main thread stack using the --main-stacksize= flag.
==12046==  The main thread stack size used in this run was 8388608.

And below with :

ogr2ogr -f filegdb toto.gdb data/poly.shp -nlt MULTIPOLYGON -a_srs EPSG:4326
==12083== Invalid write of size 1
==12083==    at 0xD32C0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD302A7D: ShapeCompressor::CompressMultipart(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD3035D1: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD38EBCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD39653C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD29BE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0x5746CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12083==    by 0x40B0D9: TranslateLayer(OGRDataSource*, OGRLayer*, OGRDataSource*, char**, char const*, int, OGRSpatialReference*, int, OGRSpatialReference*, char**, int, int, int, double, char**, long, int, OGRGeometry*, OGRGeometry*, int, char const*, char const*, int (*)(double, char const*, void*), void*) (ogr2ogr.cpp:2192)
==12083==    by 0x409485: main (ogr2ogr.cpp:1365)
==12083==  Address 0x15e9cee0 is 0 bytes after a block of size 1,024 alloc'd
==12083==    at 0x4C27878: malloc (vg_replace_malloc.c:236)
==12083==    by 0xD333F81: IOBuffer::Allocate(int, AllocationType) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD388EEA: StandardDatafile::SetupOutputBuffers(int) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD396409: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD29BE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0x5746CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12083==    by 0x40B0D9: TranslateLayer(OGRDataSource*, OGRLayer*, OGRDataSource*, char**, char const*, int, OGRSpatialReference*, int, OGRSpatialReference*, char**, int, int, int, double, char**, long, int, OGRGeometry*, OGRGeometry*, int, char const*, char const*, int (*)(double, char const*, void*), void*) (ogr2ogr.cpp:2192)
==12083==    by 0x409485: main (ogr2ogr.cpp:1365)
==12083== 
==12083== 
==12083== Process terminating with default action of signal 11 (SIGSEGV)
==12083==  Bad permissions for mapped region at address 0x16741000
==12083==    at 0xD32C0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD302A7D: ShapeCompressor::CompressMultipart(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD3035D1: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD38EBCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD39653C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD29BE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0x5746CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12083==    by 0x40B0D9: TranslateLayer(OGRDataSource*, OGRLayer*, OGRDataSource*, char**, char const*, int, OGRSpatialReference*, int, OGRSpatialReference*, char**, int, int, int, double, char**, long, int, OGRGeometry*, OGRGeometry*, int, char const*, char const*, int (*)(double, char const*, void*), void*) (ogr2ogr.cpp:2192)
==12083==    by 0x409485: main (ogr2ogr.cpp:1365)

comment:2 Changed 10 years ago by pramsey

I'm on 32-bit and don't recall that level of death and destruction... I did up the default resolution a lot so that could be the source... I'll see if I can cause it to happen here.

comment:3 Changed 10 years ago by pramsey

Correction, I'm experiencing the same death and destruction in the same place... the FGDB code just doesn't like my default precision parameters. Will have to juggle them around a bit.

Changed 10 years ago by pramsey

Attachment: fgdb-precision-2.patch added

second try

comment:4 Changed 10 years ago by pramsey

Even, this patch doesn't explode on me, running my usual processes.

comment:5 Changed 10 years ago by Even Rouault

Resolution: fixed
Status: newclosed

r22755 /trunk/gdal/ogr/ogrsf_frmts/filegdb/ (FGdbDatasource.cpp FGdbLayer.cpp drv_filegdb.html ogr_fgdb.h): FileGDB: add options for controlling precision grid (patch fgdb-precision-2.patch by Paul Ramsey, #4163)

r22756 /trunk/autotest/ogr/ogr_fgdb.py: More testing of FGDB driver

r22757 /trunk/gdal/ogr/ogrsf_frmts/filegdb/ (6 files): FileGDB: stylistic change to conform to usual GDAL coding habits : 4 space indentation level

comment:6 Changed 10 years ago by pramsey

Could you let me know before you re-format things on me, you've caught me with a bunch of changes that are now in conflict.

comment:7 Changed 10 years ago by pramsey

Incidentally it looks like most of the spacing is 2-space.

comment:8 Changed 10 years ago by Even Rouault

ah sorry. When I refer to 4 space indentation level, this is for the rest of the code base. This driver was an exception.

Note: See TracTickets for help on using tickets.