Opened 11 years ago

Closed 10 years ago

#4469 closed defect (fixed)

CSV driver writes fixed-width data for numeric fields

Reported by: cdestigter Owned by: Robert Coup
Priority: normal Milestone:
Component: OGR_SF Version: 1.9.0
Severity: normal Keywords: CSV
Cc: warmerdam

Description

If I create a CSV datasource, and write a numeric field to it with width/precision set, the value becomes padded with spaces (before the decimal point) and zeroes (after the decimal point). This is a waste of bytes...

test case:

    from osgeo import ogr, osr
    ogr.UseExceptions()
    driver = ogr.GetDriverByName('CSV')
    ds = driver.CreateDataSource('test.csv')
    srs = osr.SpatialReference()
    srs.ImportFromEPSG(4326)
    layer = ds.CreateLayer('test', srs, ogr.wkbPoint, [])

    field = ogr.FieldDefn('stringfield', ogr.OFTString)
    field.SetWidth(50)
    layer.CreateField(field)

    field = ogr.FieldDefn('numberfield', ogr.OFTReal)
    field.SetWidth(50)
    field.SetPrecision(25)
    layer.CreateField(field)

    feature = ogr.Feature(layer.GetLayerDefn())
    feature.SetField('stringfield', 'some value')
    feature.SetField('numberfield', 10.5)

    layer.CreateFeature(feature)

    layer.SyncToDisk()
    ds.Destroy()

Produces this:

stringfield,numberfield
some value,                      10.5000000000000000000000000

Change History (6)

comment:1 by cdestigter, 11 years ago

Possibly trailing zeroes make sense. In any case they're hard to remove without resorting to a string-replace.

But I fixed the leading whitespace by not specifying field width: https://github.com/craigds/osgeo-gdal/commit/0a3f0f0bb2c6880e705934f223cd29780d19b3b1

comment:2 by cdestigter, 11 years ago

Owner: changed from warmerdam to Robert Coup

comment:3 by Robert Coup, 10 years ago

Resolution: fixed
Status: newclosed

Fixed in trunk [25046] and 1.9 [25047]

comment:4 by Even Rouault, 10 years ago

Cc: warmerdam added

Robert,

I'm not sure that such a general fix (in ogrfeature.cpp) is really appropriate for 1.9 branch, especially so close from a release. Although I can't point a precise example of a situation, there are perhaps users that strongly rely on the existing formatting (for fixed size formatting for example). It would seem more prudent to restrict that to trunk.

comment:5 by Robert Coup, 10 years ago

Resolution: fixed
Status: closedreopened

Fair enough :)

Reverted from 1.9 in [25049].

Are there additional tests/places you can suggest that I dig into to make sure it doesn't break other drivers/behaviours?

comment:6 by Even Rouault, 10 years ago

Resolution: fixed
Status: reopenedclosed

I'm not sure it would break something in GDAL itself. It was just a word of caution. External code to GDAL could theoritically use OGR_F_GetFieldAsString() and rely on the fact that the string is of fixed size. On trunk, this is not very risky, but in a bug-fix release changing that behaviour could be considered as a "new feature".

Note: See TracTickets for help on using tickets.