Opened 8 years ago

Closed 7 years ago

#4452 closed defect (fixed)

OGR CSV mangles non-platform newlines in data

Reported by: cdestigter Owned by: Robert Coup
Priority: normal Milestone:
Component: OGR_SF Version: 1.9.0
Severity: normal Keywords: CSV
Cc:

Description

When writing a CSV using OGR, newlines in data get converted to local newlines.

i.e. on linux, if I write this data to a field, I get a linux newline:

feature.SetField?("testfield", "line one\r\nline two")

This is independent of the value of the LINEFORMAT config flag.

Expected behavior: I'd expect the data to be unchanged. If I write a \r\n I expect it to show up in the CSV file. It shouldn't be affected by LINEFORMAT, but it certainly shouldn't get localised.

Change History (6)

comment:2 Changed 7 years ago by cdestigter

hoping that this will get some attention, here's a test case:

from osgeo import ogr, osr
ogr.UseExceptions()
driver = ogr.GetDriverByName('CSV')
ds = driver.CreateDataSource('test.csv')
srs = osr.SpatialReference()
srs.ImportFromEPSG(4326)
layer = ds.CreateLayer('test', srs, ogr.wkbPoint, [])

field = ogr.FieldDefn('stringfield', ogr.OFTString)
field.SetWidth(50)
layer.CreateField(field)

feature = ogr.Feature(layer.GetLayerDefn())
feature.SetField('stringfield', 'windows newline:\r\nlinux newline:\nend of string:')

layer.CreateFeature(feature)

layer.SyncToDisk()
ds.Destroy()

open('test.csv').read()

Produces:

'stringfield\n"windows newline:\nlinux newline:\nend of string:"\n'

comment:3 Changed 7 years ago by Even Rouault

The code to remove the \r was written voluntary apparently. I don't understand the reason myself, but maybe the original author (Frank) will remember... ?

comment:4 Changed 7 years ago by warmerdam

I can't give a good explanation why the set name/value code normalizes newlines. I would not object to removing this.

comment:5 Changed 7 years ago by cdestigter

Owner: changed from warmerdam to Robert Coup

comment:6 Changed 7 years ago by Robert Coup

Resolution: fixed
Status: newclosed

Fixed on trunk [25044] and 1.9 [25045] with tests.

Note: See TracTickets for help on using tickets.