[raster] Redefine and expand the keywords available to MapAlgebra expressions

[dustymugs]

The goal is to rename the existing keywords to have some protection as it is possible that users may want to use these keywords.

For 1-raster: RAST

For 2-raster: RAST1 and RAST2

Also, expand the keywords available for MapAlgebra expressions.

[dustymugs]

For renaming the existing keywords for some safety, I wouldn't mind something like

_RAST_

_RAST1_

_RAST2_

[dustymugs]

As for additional keywords, I'd like to see the following in addition to the pixel value

x - pixel (column) position

y - line (row) position

ulx - upper-left corner of pixel's raster

uly - upper-left corner of pixel's raster

skewx - skew of pixel's raster

skewy - skew of pixel's raster

scalex - scale of pixel

scaley - scale of pixel

[pracine]

Wow! Do you really have an application for all those values? Isn't this going to make each pixel evaluation much slower?

[dustymugs]

For the user function variants of MapAlgebra, we should try to maintain support for 3 and 4 parameter user functions. 3 parameter functions would preserve current implementations…

userfunction(rast1value double precision, rast2value double precision, userargs text[])

4 parameter functions would add a double precision array

userfunction(rast1value double precision, rast2value double precision, userargs text[], rastattributes double precision[])

The double precision array would have 16 elements, 8 elements for each raster.

[dustymugs]

Replying to pracine:

Wow! Do you really have an application for all those values? Isn't this going to make each pixel evaluation much slower?

I personally don't. I'm just taking strk's addition of x and y to its natural conclusion.

[dustymugs]

Instead of a single dimensional array with 16 elements, maybe a two dimensional array of 8 elements.

[pracine]

I can think of applications using x and y since those are varying from one pixel to the other but since all the other values are constant from one pixel to the other one can easily use ST_MapAlgebraFct and pass those values as parameters to its custom function. This way we don't make 99% of the applications much slower.

[strk]

One idea could be prefixing with : That character is used by psql for expanding variables. Example:

echo "select :n" | psql —set n="'hello'"

So we would have:

• :rast * :rast1 * :rast2

Then I'd see those as roots for further detail, "rast" would identify the pixel taken in consideration, and you could then get more informations about it by adding a point and another label:

• :rast.val
• :rast.x
• :rast.y

Or the shorter and more underground:

• :r.v :r.x :r.y
• :r1.v :r1.x :r1.y
• :r2.v :r2.x :r2.y

Note that X and Y don't make things much slower (except in the keyword substitution phase, which can be optimized further). I also think all the other informations can be derived with a proper expression.

[dustymugs]

Replying to pracine:

I can think of applications using x and y since those are varying from one pixel to the other but since all the other values are constant from one pixel to the other one can easily use ST_MapAlgebraFct and pass those values as parameters to its custom function. This way we don't make 99% of the applications much slower.

OK. That does sound better to me. I'll add the x and y parameters to the 2-raster variants in the next several days. We should add some protection to the keywords though. Something more than just "RAST".

[dustymugs]

The use of colon looks good to me. Should it be ":rast" or ":RAST"? I prefer the upper-case ":RAST".

[strk]

I prefer writing quick and w/out pressing SHIFT too much, so my preference goes to :r.v to get value of the raster (single SHIFT for the colon, which means "warp to keywordland). Requiring a space after the keyword would also be nice so the full sequence would be 5 characters ":r.v "

Last word to Pierre, anyway

[dustymugs]

I do like the underground ":r" method. Pierre?

[pracine]

In the very beginning I was thinking about [rast].

I think we should support [rast], [rast.val], [rast.x] and [rast.y] or using your syntax: :rast, :rast.val, :rast.x, :rast.y

So [rast] and [rast.val] are equivalent and can be merged at the beginning of the function so only one is evaluated in the loop.

Personnally I prefer the brackets. Easier to read in an expression.

[pracine]

I also prefer the long 'rast' then the short 'r'…

[pracine]

This is not fortran…

[strk]

I find the brackets more readable too. Will also save a SHIFT… and I'll survive overground

[dustymugs]

brackets it is then! So, the new keywords will be

[rast] = pixel value or is this supposed to be something else?

[rast.val] = pixel value

[rast.x] = pixel (column) position

[rast.y] = line (row) position

I'll take care of the 2-raster versions.

[pracine]

Don't forget to update the scripts using ST_MapAlgebra in the script/plpgsql folder.

[strk]

@dustymugs: is there a time estimate for this ? I'd like to write an article about producing NTv2 gridshift files using PostGIS raster and willing to use the new syntax in it

[strk]

Note that #1119 is about enhancing expressivity further to take into account band number. Using [n] for addressing a band number wouldn't be that readable inside brackets. I wonder if we could find a general syntax capable of addressing multiple rasters (rather than only 1 or 2) and multiple bands in them.

Like:

:rast[1].band[2].val
:rast[2].band[1].x

Or, if we prefer square brackets:

[rast1.band2.val]
[rast2.band1.x]

With defaults:

[rast.band2.y] -- defaults to rast1
[rast3.band.val] -- defaults to band1
[rast.x] -- defaults to rast1.band1.x
[rast] -- defaults to rast1.band1.val

[dustymugs]

Replying to strk:

Note that #1119 is about enhancing expressivity further to take into account band number. Using [n] for addressing a band number wouldn't be that readable inside brackets. I wonder if we could find a general syntax capable of addressing multiple rasters (rather than only 1 or 2) and multiple bands in them.

Like:

:rast[1].band[2].val
:rast[2].band[1].x

Or, if we prefer square brackets:

[rast1.band2.val]
[rast2.band1.x]

With defaults:

[rast.band2.y] -- defaults to rast1
[rast3.band.val] -- defaults to band1
[rast.x] -- defaults to rast1.band1.x
[rast] -- defaults to rast1.band1.val

You may want to put this in the other ticket (#1119) as this feels like it is better suited there.

[pracine]

Replying to dustymugs:

Replying to strk:

Note that #1119 is about enhancing expressivity further to take into account band number. Using [n] for addressing a band number wouldn't be that readable inside brackets. I wonder if we could find a general syntax capable of addressing multiple rasters (rather than only 1 or 2) and multiple bands in them.

Like:

:rast[1].band[2].val
:rast[2].band[1].x

Or, if we prefer square brackets:

[rast1.band2.val]
[rast2.band1.x]

With defaults:

[rast.band2.y] -- defaults to rast1
[rast3.band.val] -- defaults to band1
[rast.x] -- defaults to rast1.band1.x
[rast] -- defaults to rast1.band1.val

You may want to put this in the other ticket (#1119) as this feels like it is better suited there.

And please see my comment in the other ticket.

[dustymugs]

Replying to strk:

@dustymugs: is there a time estimate for this ? I'd like to write an article about producing NTv2 gridshift files using PostGIS raster and willing to use the new syntax in it

I'll be working on the 2-raster versions today. I'm leaving the 1-raster versions for someone else, hopefully David.

[dustymugs]

Added keywords to 2-raster ST_MapAlgebraExpr in r9023.

[strk]

1 raster version implemented with r9031

[dustymugs]

Reopening as ST_MapAlgebraFct variants should also pass the position information into user functions.

[dustymugs]

Support added for 2-raster ST_MapAlgebraFct in r9052. Now just for 1-raster ST_MapAlgebraFct to have similar support.

User functions can now use the following four parameter signature in addition to the existing three parameter signature.

myfunction(double precision, double precision, int[], VARIADIC text[])

The int[] contains the x and y pixel positions of the two rasters.

0 ⇒ raster 1 x coordinate
1 ⇒ raster 1 y coordinate
2 ⇒ raster 2 x coordinate
3 ⇒ raster 2 y coordinate

[dzwarg]

Added in r9070.

[strk]

Documentation should be updated to reflect the change. See ​http://postgis.refractions.net/documentation/manual-svn/RT_ST_MapAlgebraExpr.html

[dzwarg]

Added docs in #9111