r.in.xyz not importing data when line is 800 characters long

[moreno]

When trying to import data from ASCII file using r.in.xyz, in case of too many decimal digits I get the message "not enough data columns".

r.in.xyz "input=C:/Documents and Settings/moreno/Desktop/dati.txt" method=min type=FCELL fs=\t x=1 y=2 z=15 output=import_xyz

data in attach (it's always the same number, just to show an example)

[annakrat]

It fails because the length of the line in your file exceeds the maximum buffer size (256). But of course, it should not fail when the item corresponding to the maximum column requested is still within that buffer. Maybe related to this ​discussion?

[moreno]

Replying to annakrat:

It fails because the length of the line in your file exceeds the maximum buffer size (256). But of course, it should not fail when the item corresponding to the maximum column requested is still within that buffer. Maybe related to this ​discussion?

I think it's not related, the problem with r.in.xyz occours also in the middle of the line, not at the ond of it (so, it doesn't depend on the EOL out of buffer)

[annakrat]

Replying to moreno:

Replying to annakrat:

It fails because the length of the line in your file exceeds the maximum buffer size (256). But of course, it should not fail when the item corresponding to the maximum column requested is still within that buffer. Maybe related to this ​discussion?

I think it's not related, the problem with r.in.xyz occours also in the middle of the line, not at the ond of it (so, it doesn't depend on the EOL out of buffer)

Well, you can simply test it on your data, when you make the lines in the CSV file shorter. With max 254 characters per line, it starts to work for me. The problem is that everything after the limit is then read into the next line instead of the next line itself.

[hamish]

Hi,

I have increased the buffer size to 1024 chars in the development branches r58203,4. It doesn't really fix the heart of the problem but fixes it for you and makes it less likely to occur in the future (maybe we should keep going to 4096 chars). Please test with tomorrow's nightly build of WinGrass 6.5.

When the line is longer than the specified input buffer I'm not sure if it really matters if G_getl2() flushes the rest of the line or not. Either way the data-read has gone wrong and you'll want a G_fatal_error(), so the question becomes at what point to throw that error, and what the error should say. _("Line is too long for input buffer") might be a good candidate for what it should say.

thanks, Hamish

ps- consider to use fs=tab instead of fs='\t'.

[martinl]

Ticket retargeted after milestone closed