delimited text layer plugin: robustness

[hamish]

Hi,

It seems MS Office for OSX lives in the past: When you save a .csv spreadsheet it saves it using Mac OS9 newlines, i.e. \r not \n.

Using the QGIS 0.9.2rc1 Mac binaries the delimited text layer plugin fails to parse these files correctly, it seems to get the top line column names ok, but then considers all other data lines as more top line column names.

Using nedit under X11 (I hear BBedit and a tiny drag-n-drop app called ConvertNewlines* can do it too), we resaved the file in UNIX/OSX newline format (\n) and then it worked fine.

[*] ​http://lionel.kr.fh-niederrhein.de/~dalitz/data/software/macosx/#tools

It sure would be nice if the delimited text layer plugin could be modified to handle that case. It is a strange and magical thing to try and explain newlines to a Mac using biologist, and who knows when MS will get with the program.

Radim wrote a fgets() replacement for GRASS which will handle all UNIX, Mac OS9, and DOS newline styles, G_getl2():

​http://trac.osgeo.org/grass/browser/grass/trunk/lib/gis/getl.c

Also we noticed import errors if excel put additional delimiter chars at the end of each data line. (in our case two extra commas: "lat,lon,," After regex'ing those away in nedit we could load the station locations in ok.

thanks, Hamish

[homann]

Is the newline just '\r', not '\r\n' ?

Trying to think of the downsiode with this approach...

[hamish]

Replying to homann:

Is the newline just '\r', not '\r\n' ?

Yes, the MacOS9 newline is just '\r'.

Hamish

[jef]

fixed in r14448

[homann]

I have recollections of that the fix was not as simple as I first thought. Have you tested it with umlauts and utf-8?

Maybe it was me being lazy... :-)