Opened 18 months ago

Closed 18 months ago

Last modified 18 months ago

#6356 closed defect (fixed)

s57 to postgres import fails with LNAM_REFS=on for gdal 2.0.2

Reported by: hbgdal Owned by: warmerdam
Priority: normal Milestone: 1.11.5
Component: default Version: 2.0.2
Severity: normal Keywords: lnam_refs OFTStringList s57
Cc:

Description

gdal 2.0.2 ogr2ogr fails to import attached file US5CA11M.000. gdal 1.9.2 gdal succeeds.

#step 1 Create a schema named converted in marine db. echo "drop schema converted cascade; create schema converted;" | psql marine

#Step 2. Explicitly turn on some options. export OGR_S57_OPTIONS='SPLIT_MULTIPOINT=ON,ADD_SOUNDG_DEPTH=ON,LNAM_REFS=ON'

#Step 3 Run ogr2ogr with debug on to view error messages. ogr2ogr --DEBUG ON -skipfailures -f "PostgreSQL" 'PG:dbname=marine schemas=converted' $PWD/US5CA11M.000 C_ASSO 2>&1 |tee e

Result

gdal shows following message (among others). PG: Truncated lnam_refs field value, it was too long

From postgres log.... 2016-02-08 16:30:32 PST ERROR: malformed array literal: "{"0226190A9B3100" 2016-02-08 16:30:32 PST CONTEXT: COPY c_asso, line 1, column lnam_refs: "{"0226190A9B3100" 2016-02-08 16:30:32 PST STATEMENT: COPY "c_asso" ("rcid", "prim", "grup", "objl", "rver", "agen", "fidn", "fids", "lnam", "lnam_refs", "ffpt_rind", "nobjnm", "objnam", "inform", "ninfom", "ntxtds", "picrep", "scamax", "scamin", "txtdsc", "recdat", "recind", "sordat", "sorind") FROM STDIN;


Attachments (3)

US5CA11M.000 (94.4 KB) - added by hbgdal 18 months ago.
s57 file that reproduced bug.
e (3.0 KB) - added by hbgdal 18 months ago.
Error log with the workaround fix.
diff.ogrpgdumplayer.cpp (933 bytes) - added by hbgdal 18 months ago.
workaround for the the problem with this file.

Download all attachments as: .zip

Change History (6)

Changed 18 months ago by hbgdal

Attachment: US5CA11M.000 added

s57 file that reproduced bug.

Changed 18 months ago by hbgdal

Attachment: e added

Error log with the workaround fix.

Changed 18 months ago by hbgdal

Attachment: diff.ogrpgdumplayer.cpp added

workaround for the the problem with this file.

comment:1 Changed 18 months ago by hbgdal

I am not an expert in gdal. Would really appreciate if you could comment on whether its a real bug, or some problem with my configuration. lnam is an important char[16] field in s57 specs and lam_refs is an array of char[16].

I modified the code in ogrpgdumplayer.cpp to allow lnam_refs to go beyond 16 chars. This "fixes" the problem for me for this chart.

We are trying to migrate from gdal-1.9.2 (which we have been using for more than a year).

The gdal 2.0.2 has been compiled on ubuntu 14.04. ./configure --with-libtiff=/usr/lib --with-python --with-mysql

comment:2 Changed 18 months ago by Even Rouault

Milestone: 1.11.5
Resolution: fixed
Status: newclosed

The issue is due to COPY mode (the default since GDAL 2.0) incorrectly taking into account field width with StringList? fields.

So you have 2 workarounds: either add "--config PG_USE_COPY NO", or add "-unsetFieldWidth"

Fixed by trunk r33384, branches/2.0 r33385, branches/1.11 r33386 "PG/PGDump: in copy mode (the default on layer creation), do not truncate the concatenated string list to the field width (#6356)"

comment:3 Changed 18 months ago by Even Rouault

(Note: the commit message for 1.11 r33386 is partly wrong. Copy mode is the default only since GDAL 2.0)

Note: See TracTickets for help on using tickets.