FGDB options for controlling precision grid

[pramsey]

The FGDB file creates features on a precision grid, so users should be able to modify the default grid when creating new layers. Attached patch allows this.

[Even Rouault]

Paul, I applied the patch in my dev tree, and I get really nasty crashes when running the ogr_fgdb.py test (or when doing ogr2ogr -f filegdb some.gdb some.shp). This is with Linux 64bit.

Here are the Valgrind traces with ogr_fgdb.py :

==12046==
==12046== Invalid write of size 1
==12046==    at 0xF08F0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF065547: ShapeCompressor::CompressMultipoint(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0665EA: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F1BCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F953C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xEFFEE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0x75B6CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12046==    by 0x773FD84: OGR_L_CreateFeature (ogrlayer.cpp:442)
==12046==    by 0x15953442: _wrap_Layer_CreateFeature (ogr_wrap.cpp:3431)
==12046==    by 0x4A7A06: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A9670: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==12046==    by 0x4A7808: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==  Address 0x16d260c0 is 0 bytes after a block of size 1,024 alloc'd
==12046==    at 0x4C27878: malloc (vg_replace_malloc.c:236)
==12046==    by 0xF096F81: IOBuffer::Allocate(int, AllocationType) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0EBEEA: StandardDatafile::SetupOutputBuffers(int) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F9409: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xEFFEE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0x75B6CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12046==    by 0x773FD84: OGR_L_CreateFeature (ogrlayer.cpp:442)
==12046==    by 0x15953442: _wrap_Layer_CreateFeature (ogr_wrap.cpp:3431)
==12046==    by 0x4A7A06: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A9670: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==12046==    by 0x4A7808: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A854F: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==
==12046==
==12046== More than 10000000 total errors detected.  I'm not reporting any more.
==12046== Final error counts will be inaccurate.  Go fix your program!
==12046== Rerun with --error-limit=no to disable this cutoff.  Note
==12046== that errors may occur in your program without prior warning from
==12046== Valgrind, because errors are no longer being displayed.
==12046==
==12046==
==12046== Process terminating with default action of signal 11 (SIGSEGV)
==12046==  Access not within mapped region at address 0x17B97000
==12046==    at 0xF08F0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF065547: ShapeCompressor::CompressMultipoint(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0665EA: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F1BCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F953C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xEFFEE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0x75B6CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12046==    by 0x773FD84: OGR_L_CreateFeature (ogrlayer.cpp:442)
==12046==    by 0x15953442: _wrap_Layer_CreateFeature (ogr_wrap.cpp:3431)
==12046==    by 0x4A7A06: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A9670: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==12046==    by 0x4A7808: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==  If you believe this happened as a result of a stack
==12046==  overflow in your program's main thread (unlikely but
==12046==  possible), you can try to increase the size of the
==12046==  main thread stack using the --main-stacksize= flag.
==12046==  The main thread stack size used in this run was 8388608.

And below with :

ogr2ogr -f filegdb toto.gdb data/poly.shp -nlt MULTIPOLYGON -a_srs EPSG:4326

==12083== Invalid write of size 1
==12083==    at 0xD32C0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD302A7D: ShapeCompressor::CompressMultipart(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD3035D1: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD38EBCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD39653C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD29BE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0x5746CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12083==    by 0x40B0D9: TranslateLayer(OGRDataSource*, OGRLayer*, OGRDataSource*, char**, char const*, int, OGRSpatialReference*, int, OGRSpatialReference*, char**, int, int, int, double, char**, long, int, OGRGeometry*, OGRGeometry*, int, char const*, char const*, int (*)(double, char const*, void*), void*) (ogr2ogr.cpp:2192)
==12083==    by 0x409485: main (ogr2ogr.cpp:1365)
==12083==  Address 0x15e9cee0 is 0 bytes after a block of size 1,024 alloc'd
==12083==    at 0x4C27878: malloc (vg_replace_malloc.c:236)
==12083==    by 0xD333F81: IOBuffer::Allocate(int, AllocationType) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD388EEA: StandardDatafile::SetupOutputBuffers(int) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD396409: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD29BE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0x5746CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12083==    by 0x40B0D9: TranslateLayer(OGRDataSource*, OGRLayer*, OGRDataSource*, char**, char const*, int, OGRSpatialReference*, int, OGRSpatialReference*, char**, int, int, int, double, char**, long, int, OGRGeometry*, OGRGeometry*, int, char const*, char const*, int (*)(double, char const*, void*), void*) (ogr2ogr.cpp:2192)
==12083==    by 0x409485: main (ogr2ogr.cpp:1365)
==12083== 
==12083== 
==12083== Process terminating with default action of signal 11 (SIGSEGV)
==12083==  Bad permissions for mapped region at address 0x16741000
==12083==    at 0xD32C0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD302A7D: ShapeCompressor::CompressMultipart(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD3035D1: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD38EBCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD39653C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD29BE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0x5746CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12083==    by 0x40B0D9: TranslateLayer(OGRDataSource*, OGRLayer*, OGRDataSource*, char**, char const*, int, OGRSpatialReference*, int, OGRSpatialReference*, char**, int, int, int, double, char**, long, int, OGRGeometry*, OGRGeometry*, int, char const*, char const*, int (*)(double, char const*, void*), void*) (ogr2ogr.cpp:2192)
==12083==    by 0x409485: main (ogr2ogr.cpp:1365)

[pramsey]

I'm on 32-bit and don't recall that level of death and destruction... I did up the default resolution a lot so that could be the source... I'll see if I can cause it to happen here.

[pramsey]

Correction, I'm experiencing the same death and destruction in the same place... the FGDB code just doesn't like my default precision parameters. Will have to juggle them around a bit.

[pramsey]

second try

[pramsey]

Even, this patch doesn't explode on me, running my usual processes.

[Even Rouault]

r22755 /trunk/gdal/ogr/ogrsf_frmts/filegdb/ (FGdbDatasource.cpp FGdbLayer.cpp drv_filegdb.html ogr_fgdb.h): FileGDB: add options for controlling precision grid (patch fgdb-precision-2.patch by Paul Ramsey, #4163)

r22756 /trunk/autotest/ogr/ogr_fgdb.py: More testing of FGDB driver

r22757 /trunk/gdal/ogr/ogrsf_frmts/filegdb/ (6 files): FileGDB: stylistic change to conform to usual GDAL coding habits : 4 space indentation level

[pramsey]

Could you let me know before you re-format things on me, you've caught me with a bunch of changes that are now in conflict.

[pramsey]

Incidentally it looks like most of the spacing is 2-space.

[Even Rouault]

ah sorry. When I refer to 4 space indentation level, this is for the rest of the code base. This driver was an exception.