newlines in sed scripts not portable for wingrass

[hamish]

Hi,

this is not portable for wingrass:

echo "abc" | sed -e 's|b|\n|'

on linux you get the newline:

a
c

but in WinGrass you just get a quoted "n":

anc

damage from this is only cosmetic AFAICT (r.in.wms -l without xml2), but there may be other places where it matters more.

Hamish

[glynn]

Replying to hamish:

this is not portable for wingrass:

echo "abc" | sed -e 's|b|\n|'

on linux you get the newline:

but in WinGrass you just get a quoted "n":

It shouldn't work on Linux either; the documentation makes no mention of "\n" in the replacement. If you want to insert a literal newline, you're supposed to use backslash-newline, i.e.:

echo "abc" | sed -e 's|b|\
|'

which works on both Windows and Linux, and should work according to ​POSIX.

[hamish]

Replying to glynn:

If you want to insert a literal newline, you're supposed to use backslash-newline, i.e.:

echo "abc" | sed -e 's|b|\
|'

Only gotcha is that the sed expression needs 'single' quotes not "double" ones to get that to work:

$ echo "abc" | sed -e 's|b|\  
|'
a
c

$ echo "abc" | sed -e "s|b|\ 
|"
ac

but with that it's good. thanks.

committed to 6.5svn r.in.wms with r43549, a couple of left-hand-side s/ '\n's remain, will wait and see how those behave (since sed is line by line I'm kinda surprised those would work at all even when right-hand-side replacement was functional.. shrug).

I guess the right hand side was working before because it was using "double" quotes and the shell was doing the newline replacement, not sed. Sort of like "echo -e" without asking for it..

Hamish

[glynn]

Replying to hamish:

If you want to insert a literal newline, you're supposed to use backslash-newline, i.e.:

Only gotcha is that the sed expression needs 'single' quotes not "double" ones to get that to work:

The sed expression requires backslash-newline. How to pass such an expression to sed depends upon which shell (if any) you are using. With bash, it's good practice to use single quotes unless you specifically need

committed to 6.5svn r.in.wms with r43549, a couple of left-hand-side s/ '\n's remain, will wait and see how those behave

A \n in the pattern is valid.

(since sed is line by line I'm kinda surprised those would work at all even when right-hand-side replacement was functional.. shrug).

sed commands work on the "pattern space". sed operates by reading each line into the pattern space in turn. However, certain commands (e.g. G, N) append data preceded by a literal newline to the pattern space; this is one way that newlines can get into the pattern space (substitutions can also introduce them).

A common sed idiom is "{;N;<commands>;P;D;}". N appends the next line to the pattern space preceded by a newline, P prints everything up to the first newline, D deletes everything up to the newline (and suppresses the automatic reading of the next line). This allows <commands> to work on a 2-line "window", so you can e.g. use "s" commands to match phrases which might be split over two lines.

I guess the right hand side was working before because it was using "double" quotes and the shell was doing the newline replacement, not sed. Sort of like "echo -e" without asking for it..

No, bash itself doesn't interpret \n as a newline. Without quotes, \n will evaluate to just n. With single or double quotes, it will evaluate to \n. It appears that some versions of GNU sed interpret a \n in the replacement as a newline, contrary to the documentation and the standard (and they do this even with --posix).

FWIW, sed 4.2 on Linux interprets \n, sed 3.02 on Windows doesn't. I suspect that it may be a case of Linux systems typically having newer versions of sed than MSys.

Hang on; the current sed manual says this, under "Reporting Bugs":

Here are a few commonly reported bugs that are not bugs. 
...
Regex syntax clashes (problems with backslashes)
...
In addition, this version of sed  supports several escape characters (some of which are multi-character) to insert non-printable characters in scripts (\a, \c, \d, \o, \r, \t, \v, \x). These can cause similar problems with scripts written for other seds.

[hamish]

\ backported to 6.4 in r43607.