OGR CSV mangles non-platform newlines in data

[cdestigter]

When writing a CSV using OGR, newlines in data get converted to local newlines.

i.e. on linux, if I write this data to a field, I get a linux newline:

feature.SetField?("testfield", "line one\r\nline two")

This is independent of the value of the LINEFORMAT config flag.

Expected behavior: I'd expect the data to be unchanged. If I write a \r\n I expect it to show up in the CSV file. It shouldn't be affected by LINEFORMAT, but it certainly shouldn't get localised.

[cdestigter]

Easy fix here:  https://github.com/craigds/osgeo-gdal/commit/dbbd2341f7379daa564fb329f2da8e0343e62d64

[cdestigter]

hoping that this will get some attention, here's a test case:

from osgeo import ogr, osr
ogr.UseExceptions()
driver = ogr.GetDriverByName('CSV')
ds = driver.CreateDataSource('test.csv')
srs = osr.SpatialReference()
srs.ImportFromEPSG(4326)
layer = ds.CreateLayer('test', srs, ogr.wkbPoint, [])

field = ogr.FieldDefn('stringfield', ogr.OFTString)
field.SetWidth(50)
layer.CreateField(field)

feature = ogr.Feature(layer.GetLayerDefn())
feature.SetField('stringfield', 'windows newline:\r\nlinux newline:\nend of string:')

layer.CreateFeature(feature)

layer.SyncToDisk()
ds.Destroy()

open('test.csv').read()

Produces:

'stringfield\n"windows newline:\nlinux newline:\nend of string:"\n'

[rouault]

The code to remove the \r was written voluntary apparently. I don't understand the reason myself, but maybe the original author (Frank) will remember... ?

[warmerdam]

I can't give a good explanation why the set name/value code normalizes newlines. I would not object to removing this.

[rcoup]

Fixed on trunk [25044] and 1.9 [25045] with tests.