Ticket #4163 (closed enhancement: fixed)

Opened 23 months ago

Last modified 22 months ago

FGDB options for controlling precision grid

Reported by: pramsey Owned by: rouault
Priority: high Milestone: 1.9.0
Component: default Version: svn-trunk
Severity: normal Keywords:
Cc:

Description

The FGDB file creates features on a precision grid, so users should be able to modify the default grid when creating new layers. Attached patch allows this.

Attachments

fgdb-precision.patch Download (18.2 KB) - added by pramsey 23 months ago.
fgdb-precision-2.patch Download (20.1 KB) - added by pramsey 22 months ago.
second try

Change History

Changed 23 months ago by pramsey

Changed 22 months ago by rouault

Paul, I applied the patch in my dev tree, and I get really nasty crashes when running the ogr_fgdb.py test (or when doing ogr2ogr -f filegdb some.gdb some.shp). This is with Linux 64bit.

Here are the Valgrind traces with ogr_fgdb.py : 

==12046==
==12046== Invalid write of size 1
==12046==    at 0xF08F0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF065547: ShapeCompressor::CompressMultipoint(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0665EA: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F1BCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F953C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xEFFEE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0x75B6CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12046==    by 0x773FD84: OGR_L_CreateFeature (ogrlayer.cpp:442)
==12046==    by 0x15953442: _wrap_Layer_CreateFeature (ogr_wrap.cpp:3431)
==12046==    by 0x4A7A06: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A9670: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==12046==    by 0x4A7808: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==  Address 0x16d260c0 is 0 bytes after a block of size 1,024 alloc'd
==12046==    at 0x4C27878: malloc (vg_replace_malloc.c:236)
==12046==    by 0xF096F81: IOBuffer::Allocate(int, AllocationType) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0EBEEA: StandardDatafile::SetupOutputBuffers(int) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F9409: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xEFFEE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0x75B6CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12046==    by 0x773FD84: OGR_L_CreateFeature (ogrlayer.cpp:442)
==12046==    by 0x15953442: _wrap_Layer_CreateFeature (ogr_wrap.cpp:3431)
==12046==    by 0x4A7A06: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A9670: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==12046==    by 0x4A7808: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A854F: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==
==12046==
==12046== More than 10000000 total errors detected.  I'm not reporting any more.
==12046== Final error counts will be inaccurate.  Go fix your program!
==12046== Rerun with --error-limit=no to disable this cutoff.  Note
==12046== that errors may occur in your program without prior warning from
==12046== Valgrind, because errors are no longer being displayed.
==12046==
==12046==
==12046== Process terminating with default action of signal 11 (SIGSEGV)
==12046==  Access not within mapped region at address 0x17B97000
==12046==    at 0xF08F0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF065547: ShapeCompressor::CompressMultipoint(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0665EA: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F1BCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xF0F953C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0xEFFEE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12046==    by 0x75B6CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12046==    by 0x773FD84: OGR_L_CreateFeature (ogrlayer.cpp:442)
==12046==    by 0x15953442: _wrap_Layer_CreateFeature (ogr_wrap.cpp:3431)
==12046==    by 0x4A7A06: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==    by 0x4A9670: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==12046==    by 0x4A7808: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==12046==  If you believe this happened as a result of a stack
==12046==  overflow in your program's main thread (unlikely but
==12046==  possible), you can try to increase the size of the
==12046==  main thread stack using the --main-stacksize= flag.
==12046==  The main thread stack size used in this run was 8388608.

And below with : 

ogr2ogr -f filegdb toto.gdb data/poly.shp -nlt MULTIPOLYGON -a_srs EPSG:4326

==12083== Invalid write of size 1
==12083==    at 0xD32C0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD302A7D: ShapeCompressor::CompressMultipart(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD3035D1: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD38EBCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD39653C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD29BE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0x5746CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12083==    by 0x40B0D9: TranslateLayer(OGRDataSource*, OGRLayer*, OGRDataSource*, char**, char const*, int, OGRSpatialReference*, int, OGRSpatialReference*, char**, int, int, int, double, char**, long, int, OGRGeometry*, OGRGeometry*, int, char const*, char const*, int (*)(double, char const*, void*), void*) (ogr2ogr.cpp:2192)
==12083==    by 0x409485: main (ogr2ogr.cpp:1365)
==12083==  Address 0x15e9cee0 is 0 bytes after a block of size 1,024 alloc'd
==12083==    at 0x4C27878: malloc (vg_replace_malloc.c:236)
==12083==    by 0xD333F81: IOBuffer::Allocate(int, AllocationType) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD388EEA: StandardDatafile::SetupOutputBuffers(int) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD396409: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD29BE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0x5746CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12083==    by 0x40B0D9: TranslateLayer(OGRDataSource*, OGRLayer*, OGRDataSource*, char**, char const*, int, OGRSpatialReference*, int, OGRSpatialReference*, char**, int, int, int, double, char**, long, int, OGRGeometry*, OGRGeometry*, int, char const*, char const*, int (*)(double, char const*, void*), void*) (ogr2ogr.cpp:2192)
==12083==    by 0x409485: main (ogr2ogr.cpp:1365)
==12083== 
==12083== 
==12083== Process terminating with default action of signal 11 (SIGSEGV)
==12083==  Bad permissions for mapped region at address 0x16741000
==12083==    at 0xD32C0B6: VLInt64::Compress(long, unsigned char*) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD302A7D: ShapeCompressor::CompressMultipart(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD3035D1: ShapeCompressor::Compress(unsigned char*, unsigned char*, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD38EBCD: StandardDatafile::PutValue(int, tagVARIANT&, int&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD39653C: StandardDatafile::InsertRow(FieldValue*, int&, bool, bool) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0xD29BE3B: FileGDBAPI::Table::Insert(FileGDBAPI::Row&) (in /home/even/FileGDB_API/lib/libFileGDBAPI.so)
==12083==    by 0x5746CC0: FGdbLayer::CreateFeature(OGRFeature*) (FGdbLayer.cpp:219)
==12083==    by 0x40B0D9: TranslateLayer(OGRDataSource*, OGRLayer*, OGRDataSource*, char**, char const*, int, OGRSpatialReference*, int, OGRSpatialReference*, char**, int, int, int, double, char**, long, int, OGRGeometry*, OGRGeometry*, int, char const*, char const*, int (*)(double, char const*, void*), void*) (ogr2ogr.cpp:2192)
==12083==    by 0x409485: main (ogr2ogr.cpp:1365)

Changed 22 months ago by pramsey

I'm on 32-bit and don't recall that level of death and destruction... I did up the default resolution a lot so that could be the source... I'll see if I can cause it to happen here.

Changed 22 months ago by pramsey

Correction, I'm experiencing the same death and destruction in the same place... the FGDB code just doesn't like my default precision parameters. Will have to juggle them around a bit.

Changed 22 months ago by pramsey

second try

Changed 22 months ago by pramsey

Even, this patch doesn't explode on me, running my usual processes.

Changed 22 months ago by rouault

  • status changed from new to closed
  • resolution set to fixed

r22755 /trunk/gdal/ogr/ogrsf_frmts/filegdb/ (FGdbDatasource.cpp FGdbLayer.cpp drv_filegdb.html ogr_fgdb.h): FileGDB: add options for controlling precision grid (patch fgdb-precision-2.patch by Paul Ramsey, #4163)

r22756 /trunk/autotest/ogr/ogr_fgdb.py: More testing of FGDB driver

r22757 /trunk/gdal/ogr/ogrsf_frmts/filegdb/ (6 files): FileGDB: stylistic change to conform to usual GDAL coding habits : 4 space indentation level

Changed 22 months ago by pramsey

Could you let me know before you re-format things on me, you've caught me with a bunch of changes that are now in conflict.

Changed 22 months ago by pramsey

Incidentally it looks like most of the spacing is 2-space.

Changed 22 months ago by rouault

ah sorry. When I refer to 4 space indentation level, this is for the rest of the code base. This driver was an exception.

Note: See TracTickets for help on using tickets.