Ticket #2882 (new defect)

Opened 4 years ago

Last modified 4 years ago

WideString not support by ArcSDE driver

Reported by: andersmoe Owned by: hobu
Priority: normal Milestone: 1.8.1
Component: OGR_SF Version: svn-trunk
Severity: normal Keywords: SDE arcsde widestring utf8
Cc: warmerdam, jmckenna

Description

The ArcSDE driver in OGR fails to read widestrings from the ArcSDE database. Currently only empty strings are returned .

The OFTWideString type in OGR is deprecated but proper calls to

SE_stream_get_nstring

should nevertheless be made in order to get strings of type SE_WCHAR. The attached file provides a workaround and may serve as a starting point. Please note that this does not implement UTF8 and so should not be used as-is.

Anders.

Attachments

d.txt Download (6.0 KB) - added by andersmoe 4 years ago.
arcsde widestring patch
diff.txt Download (6.7 KB) - added by andersmoe 4 years ago.
UTF16 to UTF8 recode
layer-diff.txt Download (2.7 KB) - added by andersmoe 4 years ago.
widestring diff for ogr/ogrsf_frmts/sde/ogrsdelayer.cpp
ogrsdelayer.cpp Download (89.9 KB) - added by andersmoe 4 years ago.
widestring version of ogr/ogrsf_frmts/sde/ogrsdelayer.cpp

Change History

Changed 4 years ago by andersmoe

arcsde widestring patch

  Changed 4 years ago by andersmoe

  • component changed from default to OGR_SF

  Changed 4 years ago by warmerdam

  • cc warmerdam added
  • keywords SDE added
  • owner changed from warmerdam to hobu

Howard,

Are you in a position to review, test and apply this patch?

  Changed 4 years ago by hobu

  • milestone set to 1.7.0

I am willing to take this on, but this patch will need significant revision because nstring doesn't exist in most versions of the SDK (IIRC, it is only 9.2+). #ifdef'ing the calls where appropriate (SE_WCHAR, SE_stream_get_nstring, etc) in a new patch would get it applied faster if it is a very high priority. I presume we would need some sort of iconv, etc like MapServer? has to go to UTF8 from ArcSDE's UTF16[BE|LE], but GDAL doesn't link any such thing currently.

Also, I am accumulating SDE-related bugs in my "slow" queue for now as I don't have very strong incentive to overcome the pain of doing ArcSDE stuff. MapServer?'s ArcSDE driver will need some attention soon, as will adding masks to the raster driver, so I do hope to have this issue cured by 1.7.

follow-up: ↓ 5   Changed 4 years ago by warmerdam

I would note that since 1.6 GDAL has the CPLRecode() and related functions declared in cpl_string.h.

Changed 4 years ago by andersmoe

UTF16 to UTF8 recode

in reply to: ↑ 4   Changed 4 years ago by andersmoe

Great, thanks. I simplified the fix a lot (see diff.txt); the UTF16 read from SDE is now converted to UTF8 and inserted as OFTString (also, the OGRType is set to OFTString , not OFTWideString, to keep ogrsdefeature.cpp happy, which deals only with OFTString).

I also added SE_stream_set_nstring in the write section, but I havent been able to test this yet.

  Changed 4 years ago by andersmoe

Should add that the read seems to work quite nicely, also picking up the last letters of the Norwegian alphabet.

  Changed 4 years ago by jmckenna

  • cc jmckenna added

follow-up: ↓ 9   Changed 4 years ago by jmckenna

I've tried to apply both of these patches but they both fail (missing chunks) with gdal-trunk. Should they both work with trunk?

Changed 4 years ago by andersmoe

widestring diff for ogr/ogrsf_frmts/sde/ogrsdelayer.cpp

in reply to: ↑ 8   Changed 4 years ago by andersmoe

I've attached a new diff for ogr/ogrsf_frmts/sde/ogrsdelayer.cpp called layer-diff.txt as well as my version of ogrsdelayer.cpp. From 1.6.x

Replying to jmckenna:

I've tried to apply both of these patches but they both fail (missing chunks) with gdal-trunk. Should they both work with trunk?

Changed 4 years ago by andersmoe

widestring version of ogr/ogrsf_frmts/sde/ogrsdelayer.cpp

Note: See TracTickets for help on using tickets.