Various errors loading MapInfo tables with zero length char() fields into PostGIS

[lpercich]

I have two MapInfo tables, one with and the other without geometric column. Both have some char() fields with some zero-length string (empty) values. The rows have been edited to test different combinations of empty char values.

ogr2ogr gives a "Segmentation fault" when trying to convert the MapInfo geographic table with zero-length char() fields into PGDump format using --config PG_USE_COPY=NO. To test it, change the option in test_ogr_PostgreSQL_cpemptystr.sh:

Apr 14 13:58:44 GOLEM kernel: [15623.141621] ogr2ogr[13880]: segfault at 0 ip 00007f6f465ba052 sp 00007fffe3aaf938 error 4 in libc-2.11.1.so[7f6f46537000+17a000]

The conversion of non-geographic tables with the same settings produces valid SQL: insert into ... values ('',...).

All the other following errors occur with --config PG_USE_COPY=YES, and are PostGIS errors like "missing data for column <col>" or "invalid input syntax for integer:".

PostGIS driver: the error seems to occur only in non-geographic tables, when the last char() column has an empty value. This happens even if there are other non-char columns after it. Geographic tables seem to work ok even when there is an empty char() column after the geometric column.

PGDump driver: the error seems to occur when table has a char() column as the first column having some empty values. The initial \t separator is missing: see PlineTest.sql (row 5) and NoGeomTest.sql (row 4)

I'm building from trunk on Ubuntu 10.04 64 bit

[lpercich]

Test data (MapInfo tables with empty char() fiels), test script, error log, generated SQL files

[Even Rouault]

Problems confirmed. Thanks for the quality of your analysis and reporting.

I've attached a patch that should correct the crash for sure, but for the problems related to character fields with 0 character, I'm not so sure. As it is tested only against with pgdump, but not against a live pg instance, I don't commit it yet for now. If you have the opportunity to test it and report, that would be great. Otherwise I'll do it later tonight.

[lpercich]

Replying to rouault:

Problems confirmed. Thanks for the quality of your analysis and reporting.

I've attached a patch that should correct the crash for sure, but for the problems related to character fields with 0 character, I'm not so sure. As it is tested only against with pgdump, but not against a live pg instance, I don't commit it yet for now. If you have the opportunity to test it and report, that would be great. Otherwise I'll do it later tonight.

Sure I can test it. But I have problems applying the patch (It's the first time I use the patch command). Can you please give me some suggestions?

sigfrido@GOLEM:~/cpp/gdal/ogr/ogrsf_frmts$ patch -p0 --dry-run < ticket4040.patch 
(Stripping trailing CRs from patch.)
patching file pg/ogrpgtablelayer.cpp
(Stripping trailing CRs from patch.)
patching file pgdump/ogrpgdumplayer.cpp
Hunk #1 FAILED at 243.
Hunk #2 FAILED at 276.
Hunk #3 FAILED at 307.
Hunk #4 FAILED at 407.
Hunk #5 FAILED at 506.
5 out of 5 hunks FAILED -- saving rejects to file pgdump/ogrpgdumplayer.cpp.rej

[Even Rouault]

Is your base version up-to-date to latest SVN ? Or it might be an end-of-line character issue because I've generated the patch on a Windows machine. Perhaps trying dos2unix on the patch file before might help

[lpercich]

I've done again a svn checkout from trunk:

svn checkout ​https://svn.osgeo.org/gdal/trunk/gdal gdal

and converted the patch file with fromdos (dos2unix), but with no results. Still the same error message.

[Even Rouault]

ok, don't loose your time with that. I'll retry on my nominal dev environment. The patch might have been incorrectly generated.

[Even Rouault]

r22157 /trunk/ (4 files in 3 dirs): PGDump: fix crash when inserting a feature with a geometry in a layer with a geom type of wkbNone; PG and PGDump: fix insertion of features with first field being a 0-charachter string in a non-spatial table and without FID in COPY mode

r22158 /branches/1.8/gdal/ogr/ogrsf_frmts/ (pg/ogrpgtablelayer.cpp pgdump/ogrpgdumplayer.cpp): PGDump: fix crash when inserting a feature with a geometry in a layer with a geom type of wkbNone; PG and PGDump: fix insertion of features with first field being a 0-charachter string in a non-spatial table and without FID in COPY mode

[lpercich]

Replying to rouault:

r22157 /trunk/ (4 files in 3 dirs): PGDump: fix crash when inserting a feature with a geometry in a layer with a geom type of wkbNone; PG and PGDump: fix insertion of features with first field being a 0-charachter string in a non-spatial table and without FID in COPY mode

Thank you Even, all tests run fine. I was able to load all my 104 MapInfo tables in a PostGIS DB using ogr2ogr with the different options (pg/pgdump, copy/insert). Keep up with the great work.

Sig