Opened 15 years ago

Closed 6 years ago

#2882 closed defect (wontfix)

WideString not support by ArcSDE driver

Reported by: andersmoe Owned by: hobu
Priority: normal Milestone:
Component: OGR_SF Version: svn-trunk
Severity: normal Keywords: SDE arcsde widestring utf8
Cc: warmerdam, Jeff McKenna

Description

The ArcSDE driver in OGR fails to read widestrings from the ArcSDE database. Currently only empty strings are returned .

The OFTWideString type in OGR is deprecated but proper calls to

SE_stream_get_nstring

should nevertheless be made in order to get strings of type SE_WCHAR. The attached file provides a workaround and may serve as a starting point. Please note that this does not implement UTF8 and so should not be used as-is.

Anders.

Attachments (4)

d.txt (6.0 KB ) - added by andersmoe 15 years ago.
arcsde widestring patch
diff.txt (6.7 KB ) - added by andersmoe 15 years ago.
UTF16 to UTF8 recode
layer-diff.txt (2.7 KB ) - added by andersmoe 15 years ago.
widestring diff for ogr/ogrsf_frmts/sde/ogrsdelayer.cpp
ogrsdelayer.cpp (89.9 KB ) - added by andersmoe 15 years ago.
widestring version of ogr/ogrsf_frmts/sde/ogrsdelayer.cpp

Download all attachments as: .zip

Change History (16)

by andersmoe, 15 years ago

Attachment: d.txt added

arcsde widestring patch

comment:1 by andersmoe, 15 years ago

Component: defaultOGR_SF

comment:2 by warmerdam, 15 years ago

Cc: warmerdam added
Keywords: SDE added
Owner: changed from warmerdam to hobu

Howard,

Are you in a position to review, test and apply this patch?

comment:3 by hobu, 15 years ago

Milestone: 1.7.0

I am willing to take this on, but this patch will need significant revision because nstring doesn't exist in most versions of the SDK (IIRC, it is only 9.2+). #ifdef'ing the calls where appropriate (SE_WCHAR, SE_stream_get_nstring, etc) in a new patch would get it applied faster if it is a very high priority. I presume we would need some sort of iconv, etc like MapServer has to go to UTF8 from ArcSDE's UTF16[BE|LE], but GDAL doesn't link any such thing currently.

Also, I am accumulating SDE-related bugs in my "slow" queue for now as I don't have very strong incentive to overcome the pain of doing ArcSDE stuff. MapServer's ArcSDE driver will need some attention soon, as will adding masks to the raster driver, so I do hope to have this issue cured by 1.7.

comment:4 by warmerdam, 15 years ago

I would note that since 1.6 GDAL has the CPLRecode() and related functions declared in cpl_string.h.

by andersmoe, 15 years ago

Attachment: diff.txt added

UTF16 to UTF8 recode

in reply to:  4 comment:5 by andersmoe, 15 years ago

Great, thanks. I simplified the fix a lot (see diff.txt); the UTF16 read from SDE is now converted to UTF8 and inserted as OFTString (also, the OGRType is set to OFTString , not OFTWideString, to keep ogrsdefeature.cpp happy, which deals only with OFTString).

I also added SE_stream_set_nstring in the write section, but I havent been able to test this yet.

comment:6 by andersmoe, 15 years ago

Should add that the read seems to work quite nicely, also picking up the last letters of the Norwegian alphabet.

comment:7 by Jeff McKenna, 15 years ago

Cc: Jeff McKenna added

comment:8 by Jeff McKenna, 15 years ago

I've tried to apply both of these patches but they both fail (missing chunks) with gdal-trunk. Should they both work with trunk?

by andersmoe, 15 years ago

Attachment: layer-diff.txt added

widestring diff for ogr/ogrsf_frmts/sde/ogrsdelayer.cpp

in reply to:  8 comment:9 by andersmoe, 15 years ago

I've attached a new diff for ogr/ogrsf_frmts/sde/ogrsdelayer.cpp called layer-diff.txt as well as my version of ogrsdelayer.cpp. From 1.6.x

Replying to jmckenna:

I've tried to apply both of these patches but they both fail (missing chunks) with gdal-trunk. Should they both work with trunk?

by andersmoe, 15 years ago

Attachment: ogrsdelayer.cpp added

widestring version of ogr/ogrsf_frmts/sde/ogrsdelayer.cpp

comment:10 by Jukka Rahkonen, 9 years ago

Is this ticket still valid? Do I understand right that "widestring" means this (from http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//002q00000068000000):

ArcSDE data type: SE_NSTRING_TYPE 
SQL Server data type: NCHAR or NVARCHAR
ArcGIS data type: TEXT
Notes: Used for Unicode text, up to 4,000 characters

In the documentation of GDAL SDE driver http://www.gdal.org/drv_sde.html it appears that GDAL can be set to write NSTRING fields:

"USE_NSTRING: If this option is set to "TRUE" then string fields will be created as type NSTRING. This option was added for GDAL/OGR 1.9.0."

If NSTRINGS can be written then perhaps they can be read as well and this ticket could be closed as fixed.

comment:11 by Even Rouault, 9 years ago

Milestone: 1.8.1

Removing obsolete milestone

comment:12 by Jukka Rahkonen, 6 years ago

Resolution: wontfix
Status: newclosed

Quick analysis: if not yet fixed then won't fix.

Note: See TracTickets for help on using tickets.