Opened 6 years ago

Last modified 5 years ago

#2125 assigned defect

r.in.xyz not importing data when line is 800 characters long

Reported by: moreno Owned by: hamish
Priority: major Milestone: 6.4.6
Component: Raster Version: 6.4.3
Keywords: r.in.xyz Cc: grass-dev@…
CPU: x86-64 Platform: All

Description

When trying to import data from ASCII file using r.in.xyz, in case of too many decimal digits I get the message "not enough data columns".

r.in.xyz "input=C:/Documents and Settings/moreno/Desktop/dati.txt" method=min type=FCELL fs=\t x=1 y=2 z=15 output=import_xyz

data in attach (it's always the same number, just to show an example)

Attachments (1)

dati.txt (831 bytes) - added by moreno 6 years ago.

Download all attachments as: .zip

Change History (9)

Changed 6 years ago by moreno

Attachment: dati.txt added

comment:1 Changed 6 years ago by annakrat

Milestone: 6.4.36.4.4
Platform: MSWindows XPAll
Priority: criticalmajor

It fails because the length of the line in your file exceeds the maximum buffer size (256). But of course, it should not fail when the item corresponding to the maximum column requested is still within that buffer. Maybe related to this discussion?

comment:2 in reply to:  1 ; Changed 6 years ago by moreno

Replying to annakrat:

It fails because the length of the line in your file exceeds the maximum buffer size (256). But of course, it should not fail when the item corresponding to the maximum column requested is still within that buffer. Maybe related to this discussion?

I think it's not related, the problem with r.in.xyz occours also in the middle of the line, not at the ond of it (so, it doesn't depend on the EOL out of buffer)

comment:3 in reply to:  2 Changed 6 years ago by annakrat

Replying to moreno:

Replying to annakrat:

It fails because the length of the line in your file exceeds the maximum buffer size (256). But of course, it should not fail when the item corresponding to the maximum column requested is still within that buffer. Maybe related to this discussion?

I think it's not related, the problem with r.in.xyz occours also in the middle of the line, not at the ond of it (so, it doesn't depend on the EOL out of buffer)

Well, you can simply test it on your data, when you make the lines in the CSV file shorter. With max 254 characters per line, it starts to work for me. The problem is that everything after the limit is then read into the next line instead of the next line itself.

comment:4 Changed 6 years ago by hamish

Cc: grass-dev@… added
Owner: changed from grass-dev@… to hamish
Status: newassigned

Hi,

I have increased the buffer size to 1024 chars in the development branches r58203,4. It doesn't really fix the heart of the problem but fixes it for you and makes it less likely to occur in the future (maybe we should keep going to 4096 chars). Please test with tomorrow's nightly build of WinGrass 6.5.

When the line is longer than the specified input buffer I'm not sure if it really matters if G_getl2() flushes the rest of the line or not. Either way the data-read has gone wrong and you'll want a G_fatal_error(), so the question becomes at what point to throw that error, and what the error should say. _("Line is too long for input buffer") might be a good candidate for what it should say.

thanks, Hamish

ps- consider to use fs=tab instead of fs='\t'.

comment:5 Changed 6 years ago by hamish

Summary: r.in.xyz not importing data with 5 decimal digitsr.in.xyz not importing data when line is 800 characters long

comment:6 Changed 6 years ago by hamish

Milestone: 6.4.46.4.5

comment:7 Changed 5 years ago by martinl

Milestone: 6.4.5

Ticket retargeted after milestone closed

comment:8 Changed 5 years ago by martinl

Milestone: 6.4.6
Note: See TracTickets for help on using tickets.