Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#4040 closed defect (fixed)

Various errors loading MapInfo tables with zero length char() fields into PostGIS

Reported by: lpercich Owned by: Even Rouault
Priority: normal Milestone: 1.8.1
Component: OGR_SF Version: 1.8.0
Severity: normal Keywords: PostGIS, PGDump, MapInfo
Cc:

Description

I have two MapInfo? tables, one with and the other without geometric column. Both have some char() fields with some zero-length string (empty) values. The rows have been edited to test different combinations of empty char values.

ogr2ogr gives a "Segmentation fault" when trying to convert the MapInfo? geographic table with zero-length char() fields into PGDump format using --config PG_USE_COPY=NO. To test it, change the option in test_ogr_PostgreSQL_cpemptystr.sh:

Apr 14 13:58:44 GOLEM kernel: [15623.141621] ogr2ogr[13880]: segfault at 0 ip 00007f6f465ba052 sp 00007fffe3aaf938 error 4 in libc-2.11.1.so[7f6f46537000+17a000]

The conversion of non-geographic tables with the same settings produces valid SQL: insert into ... values ('',...).

All the other following errors occur with --config PG_USE_COPY=YES, and are PostGIS errors like "missing data for column <col>" or "invalid input syntax for integer:".

PostGIS driver: the error seems to occur only in non-geographic tables, when the last char() column has an empty value. This happens even if there are other non-char columns after it. Geographic tables seem to work ok even when there is an empty char() column after the geometric column.

PGDump driver: the error seems to occur when table has a char() column as the first column having some empty values. The initial \t separator is missing: see PlineTest?.sql (row 5) and NoGeomTest?.sql (row 4)

I'm building from trunk on Ubuntu 10.04 64 bit

Attachments (2)

test_OGR_PostgreSQL_cpemptystr.zip (10.8 KB) - added by lpercich 7 years ago.
Test data (MapInfo? tables with empty char() fiels), test script, error log, generated SQL files
ticket4040.patch (2.8 KB) - added by Even Rouault 7 years ago.

Download all attachments as: .zip

Change History (9)

Changed 7 years ago by lpercich

Test data (MapInfo? tables with empty char() fiels), test script, error log, generated SQL files

comment:1 Changed 7 years ago by Even Rouault

Owner: changed from warmerdam to Even Rouault

Problems confirmed. Thanks for the quality of your analysis and reporting.

I've attached a patch that should correct the crash for sure, but for the problems related to character fields with 0 character, I'm not so sure. As it is tested only against with pgdump, but not against a live pg instance, I don't commit it yet for now. If you have the opportunity to test it and report, that would be great. Otherwise I'll do it later tonight.

Changed 7 years ago by Even Rouault

Attachment: ticket4040.patch added

comment:2 in reply to:  1 Changed 7 years ago by lpercich

Replying to rouault:

Problems confirmed. Thanks for the quality of your analysis and reporting.

I've attached a patch that should correct the crash for sure, but for the problems related to character fields with 0 character, I'm not so sure. As it is tested only against with pgdump, but not against a live pg instance, I don't commit it yet for now. If you have the opportunity to test it and report, that would be great. Otherwise I'll do it later tonight.

Sure I can test it. But I have problems applying the patch (It's the first time I use the patch command). Can you please give me some suggestions?

sigfrido@GOLEM:~/cpp/gdal/ogr/ogrsf_frmts$ patch -p0 --dry-run < ticket4040.patch 
(Stripping trailing CRs from patch.)
patching file pg/ogrpgtablelayer.cpp
(Stripping trailing CRs from patch.)
patching file pgdump/ogrpgdumplayer.cpp
Hunk #1 FAILED at 243.
Hunk #2 FAILED at 276.
Hunk #3 FAILED at 307.
Hunk #4 FAILED at 407.
Hunk #5 FAILED at 506.
5 out of 5 hunks FAILED -- saving rejects to file pgdump/ogrpgdumplayer.cpp.rej

comment:3 Changed 7 years ago by Even Rouault

Is your base version up-to-date to latest SVN ? Or it might be an end-of-line character issue because I've generated the patch on a Windows machine. Perhaps trying dos2unix on the patch file before might help

comment:4 Changed 7 years ago by lpercich

I've done again a svn checkout from trunk:

svn checkout https://svn.osgeo.org/gdal/trunk/gdal gdal

and converted the patch file with fromdos (dos2unix), but with no results. Still the same error message.

comment:5 Changed 7 years ago by Even Rouault

ok, don't loose your time with that. I'll retry on my nominal dev environment. The patch might have been incorrectly generated.

comment:6 Changed 7 years ago by Even Rouault

Milestone: 1.8.1
Resolution: fixed
Status: newclosed
Version: svn-trunk1.8.0

r22157 /trunk/ (4 files in 3 dirs): PGDump: fix crash when inserting a feature with a geometry in a layer with a geom type of wkbNone; PG and PGDump: fix insertion of features with first field being a 0-charachter string in a non-spatial table and without FID in COPY mode

r22158 /branches/1.8/gdal/ogr/ogrsf_frmts/ (pg/ogrpgtablelayer.cpp pgdump/ogrpgdumplayer.cpp): PGDump: fix crash when inserting a feature with a geometry in a layer with a geom type of wkbNone; PG and PGDump: fix insertion of features with first field being a 0-charachter string in a non-spatial table and without FID in COPY mode

comment:7 in reply to:  6 Changed 7 years ago by lpercich

Replying to rouault:

r22157 /trunk/ (4 files in 3 dirs): PGDump: fix crash when inserting a feature with a geometry in a layer with a geom type of wkbNone; PG and PGDump: fix insertion of features with first field being a 0-charachter string in a non-spatial table and without FID in COPY mode

Thank you Even, all tests run fine. I was able to load all my 104 MapInfo? tables in a PostGIS DB using ogr2ogr with the different options (pg/pgdump, copy/insert). Keep up with the great work.

Sig

Note: See TracTickets for help on using tickets.