Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#4040 closed defect (fixed)

Various errors loading MapInfo tables with zero length char() fields into PostGIS

Reported by: lpercich Owned by: Even Rouault
Priority: normal Milestone: 1.8.1
Component: OGR_SF Version: 1.8.0
Severity: normal Keywords: PostGIS, PGDump, MapInfo
Cc:

Description

I have two MapInfo? tables, one with and the other without geometric column. Both have some char() fields with some zero-length string (empty) values. The rows have been edited to test different combinations of empty char values.

ogr2ogr gives a "Segmentation fault" when trying to convert the MapInfo? geographic table with zero-length char() fields into PGDump format using --config PG_USE_COPY=NO. To test it, change the option in test_ogr_PostgreSQL_cpemptystr.sh:

Apr 14 13:58:44 GOLEM kernel: [15623.141621] ogr2ogr[13880]: segfault at 0 ip 00007f6f465ba052 sp 00007fffe3aaf938 error 4 in libc-2.11.1.so[7f6f46537000+17a000]

The conversion of non-geographic tables with the same settings produces valid SQL: insert into ... values ('',...).

All the other following errors occur with --config PG_USE_COPY=YES, and are PostGIS errors like "missing data for column <col>" or "invalid input syntax for integer:".

PostGIS driver: the error seems to occur only in non-geographic tables, when the last char() column has an empty value. This happens even if there are other non-char columns after it. Geographic tables seem to work ok even when there is an empty char() column after the geometric column.

PGDump driver: the error seems to occur when table has a char() column as the first column having some empty values. The initial \t separator is missing: see PlineTest?.sql (row 5) and NoGeomTest?.sql (row 4)

I'm building from trunk on Ubuntu 10.04 64 bit

Attachments (2)

test_OGR_PostgreSQL_cpemptystr.zip (10.8 KB) - added by lpercich 10 years ago.
Test data (MapInfo? tables with empty char() fiels), test script, error log, generated SQL files
ticket4040.patch (2.8 KB) - added by Even Rouault 10 years ago.

Download all attachments as: .zip

Change History (9)

Changed 10 years ago by lpercich

Test data (MapInfo? tables with empty char() fiels), test script, error log, generated SQL files

comment:1 Changed 10 years ago by Even Rouault

Owner: changed from warmerdam to Even Rouault

Problems confirmed. Thanks for the quality of your analysis and reporting.

I've attached a patch that should correct the crash for sure, but for the problems related to character fields with 0 character, I'm not so sure. As it is tested only against with pgdump, but not against a live pg instance, I don't commit it yet for now. If you have the opportunity to test it and report, that would be great. Otherwise I'll do it later tonight.

Changed 10 years ago by Even Rouault

Attachment: ticket4040.patch added

comment:2 in reply to:  1 Changed 10 years ago by lpercich

Replying to rouault:

Problems confirmed. Thanks for the quality of your analysis and reporting.

I've attached a patch that should correct the crash for sure, but for the problems related to character fields with 0 character, I'm not so sure. As it is tested only against with pgdump, but not against a live pg instance, I don't commit it yet for now. If you have the opportunity to test it and report, that would be great. Otherwise I'll do it later tonight.

Sure I can test it. But I have problems applying the patch (It's the first time I use the patch command). Can you please give me some suggestions?

sigfrido@GOLEM:~/cpp/gdal/ogr/ogrsf_frmts$ patch -p0 --dry-run < ticket4040.patch 
(Stripping trailing CRs from patch.)
patching file pg/ogrpgtablelayer.cpp
(Stripping trailing CRs from patch.)
patching file pgdump/ogrpgdumplayer.cpp
Hunk #1 FAILED at 243.
Hunk #2 FAILED at 276.
Hunk #3 FAILED at 307.
Hunk #4 FAILED at 407.
Hunk #5 FAILED at 506.
5 out of 5 hunks FAILED -- saving rejects to file pgdump/ogrpgdumplayer.cpp.rej

comment:3 Changed 10 years ago by Even Rouault

Is your base version up-to-date to latest SVN ? Or it might be an end-of-line character issue because I've generated the patch on a Windows machine. Perhaps trying dos2unix on the patch file before might help

comment:4 Changed 10 years ago by lpercich

I've done again a svn checkout from trunk:

svn checkout https://svn.osgeo.org/gdal/trunk/gdal gdal

and converted the patch file with fromdos (dos2unix), but with no results. Still the same error message.

comment:5 Changed 10 years ago by Even Rouault

ok, don't loose your time with that. I'll retry on my nominal dev environment. The patch might have been incorrectly generated.

comment:6 Changed 10 years ago by Even Rouault

Milestone: 1.8.1
Resolution: fixed
Status: newclosed
Version: svn-trunk1.8.0

r22157 /trunk/ (4 files in 3 dirs): PGDump: fix crash when inserting a feature with a geometry in a layer with a geom type of wkbNone; PG and PGDump: fix insertion of features with first field being a 0-charachter string in a non-spatial table and without FID in COPY mode

r22158 /branches/1.8/gdal/ogr/ogrsf_frmts/ (pg/ogrpgtablelayer.cpp pgdump/ogrpgdumplayer.cpp): PGDump: fix crash when inserting a feature with a geometry in a layer with a geom type of wkbNone; PG and PGDump: fix insertion of features with first field being a 0-charachter string in a non-spatial table and without FID in COPY mode

comment:7 in reply to:  6 Changed 10 years ago by lpercich

Replying to rouault:

r22157 /trunk/ (4 files in 3 dirs): PGDump: fix crash when inserting a feature with a geometry in a layer with a geom type of wkbNone; PG and PGDump: fix insertion of features with first field being a 0-charachter string in a non-spatial table and without FID in COPY mode

Thank you Even, all tests run fine. I was able to load all my 104 MapInfo? tables in a PostGIS DB using ogr2ogr with the different options (pg/pgdump, copy/insert). Keep up with the great work.

Sig

Note: See TracTickets for help on using tickets.