| Version 42 (modified by , 13 years ago) ( diff ) |
|---|
PostGIS Raster Working Specifications for Future Versions
Quick Links
Objective FV.01 - Being able to return a JPEG, a TIFF, a PNG or any image format supported by GDAL.
ST_bytea(raster, integer) → raster — the integer parameters is the band number of the raster.
What is does?
Open Question: When exporting a multiband raster to JPEG, TIFF, PNG, SVG or KML, how should we specify the band number in the exporting function.
There is two options to select the band to convert from a multiband raster in all the ST_AsFormat functions.
- Precede each call with ST_Band() to return a selected band.
Pros: This is a general function that can be called before any function that would otherwise require a band parameter.
Cons: This implies creating a temporary raster. This might be more elegant and general but is this too much overhead comparing with having a band parameter?
- Add a band parameter to each ST_AsFormat function.
Pros: Hypothetically less overhead.
Cons: Every functions implying access to a band should then have this parameter when in most case it would be equal to 1. In many cases it makes no sence to have to specify a band parameter since it is the whole raster that we want to export, including all the bands.
Pierre: More I think about it more I think that the first option is the best one…
mloskot: Perhaps there is a compromise in form of two sets of functions: 1) ST_As* which always burn the whole raster (all bands) 2) ST_BandAs* which takes number of band as a parameter and return only this requested band.
ST_Band(raster, integer) → raster — the integer parameters are the band number of the rasters.
Return a single band from a multiband raster. If "band" is greater than the value returned by ST_GetNumBands(), the function returns the last band. This function should be used to select a band before converting it to JPEG, TIFF, PNG, SVG or KML with the corresponding function. e.g. ST_AsTIFF(ST_Band(raster, band))
ST_AsJPEG(raster, quality) → JPEG as "bytea"
Return the raster as a JPEG encoded as a PostgreSQL bytea. By default quality is set to 75, but this option can be used to select other values. Values must be in the range 10-100. Low values result in higher compression ratios, but poorer image quality. Values above 95 are not meaningfully better quality but can but substantially larger. (copied from http://www.gdal.org/frmt_jpeg.html)
Open Question: Is JPEG export limited to raster having 8 bit unsigned integer pixeltype (8BUI)?
See how GDAL do it. It converts only 8 bits rasters. Should we do the same?
Otherwise, how do we convert other types to 8BUI? e.g. 16BUI or 8BSI?
Pierre: It might be more simple to ignore pixeltypes other than 8BUI but it would be very convenient to have a way to quickly export elevation data for example as a JPEG. It would be nice to have an elegant solution to this. Maybe something inspired from MapServer.
Proposition one (Pierre): ST_AsJPEG could simply (optionally when the pixeltype is not 8BUI) map the ST_Maximum() and ST_Minimum() value to 0-255. ST_Maximum() and ST_Minimum() are not in the spec yet but this could be on nice usage of it. They will imply caching the min and max when importing and editing. Both function should ignore the NoDataValues. They could also be two parameters passed to ST_AsJPEG(raster, quality, min, max).
Proposition two: There could also be just one parameter (string) defining a mapping method:
- Method "None": No mapping. This is possible only for 8BUI.
- Method "MaxMinValue": Use the Max and the Min cached in the raster. e.g. for 16BSI (min, max) → (-2033, 2456) → (round((-2033 - -2033)/(2456 - -2033)*255), round((2456 - -2033)/(2456 - -2033)*255)) → (0, 255).
This is equivalent to ST_AsJPEG(raster, quality, ST_Minimum(rast), ST_Maximum(rast))
- Method "MaxMinType": Use the Max and the Min allowed by the type. e.g. for 16BSI (min, max) → (-2033, 2456) → (round((-2033 - -32768)/(32767 - -32768)*255), round((2456 - -32768)/(32767 - -32768)*255)) → (120, 137)
This would be equivalent to ST_AsJPEG(raster, quality, ST_BandPixelTypeMin(rast), ST_BandPixelTypeMax(rast)). Both functions (ST_BandPixelTypeMin & ST_BandPixelTypeMax) are not yet planned and I could not find an SQL query that returns the equivalent range for a type. One possible solution.
mloskot: ATM, I have no thoughts on this issue.
Open Question: Is JPEG export limited to raster having 1 or 3 bands?
See how GDAL do it. It converts only 1 or 3 band rasters. Should we do the same? In this case 1 band rasters would be exported as a greyscale JPEG having R G and B identical and 3 band rasters would be interpreted as R, G and B.
Pierre: I think the answer should be yes. I don't see how we could have a 2 band raster fit into RGB.
mloskot: I agree, the answer should be yes.
Here is an attempt to define the different versions of the function:
The most minimalistic versions of the function should assume band 1, 2 and 3 as being r, g, b and the quality equal to 75:
ST_AsJPEG(raster) -quality = 75
A variant allow specifying the quality:
ST_AsJPEG(raster, integer)
Another variant should enable us to specify which band correspond to the r, the g and the b:
ST_AsJPEG(raster, integer, integer, integer) - raster, rband, gband, bband, quality=75
ST_AsJPEG(raster, integer, integer, integer, integer) - raster, rband, gband, bband, quality
Another version should be designed to be used with a future ST_Band(raster) function. In this case there is no attempt to extract r, g or b band from any passed raster:
ST_AsJPEG(raster, raster, raster)
ST_AsJPEG(raster, raster, raster, integer) -with the quality param
Another series should allow converting 1 band raster with pixel of type 8BUI to a grayscale JPEG (Carefull study of the GDAL behavior when converting a single band to JPEG should be done before confirming these functions):
ST_AsJPEG(raster, "GRAYSCALE") - convert only band 1 with quality = 75
ST_AsJPEG(raster, "GRAYSCALE", integer) - convert only band 1 with specified quality
ST_AsJPEG(raster, integer, "GRAYSCALE") - allow specifying the band number to convert
ST_AsJPEG(raster, integer, "GRAYSCALE", integer) - allow specifying the band number to convert and the quality
Another series should allow converting 1 band raster of ANY pixel type to a grayscale JPEG. Pixel types different than 8BUI should be mapped according to specified min, max values and a mapping mode: "MaxMinValue" (default) or "MaxMinType".
ST_AsJPEG(raster, "GRAYSCALE", min, max, text) - convert only band 1 with quality = 75
ST_AsJPEG(raster, "GRAYSCALE", integer, min, max, text) - convert only band 1 with specified quality
ST_AsJPEG(raster, integer, "GRAYSCALE", min, max, text) - allow specifying the band number to convert
ST_AsJPEG(raster, integer, "GRAYSCALE", integer, min, max, text) - allow specifying the band number to convert and the quality
ST_AsTIFF(raster, compression) → TIFF as "bytea"
Return the raster as a JPEG encoded as a PostgreSQL bytea. If raster is a multiband raster and no band were selected with ST_Band() every band are written to the resulting TIFF.
compression=[JPEG/LZW/PACKBITS/DEFLATE/CCITTRLE/CCITTFAX3/CCITTFAX4/NONE]: Set the type of compression to use. None is the default. The CCITT compression should only be used with 1bit (NBITS=1) data. JPEG should only be used with Byte data. When using JPEG add a number specifying the quality. 75 is the default. e.g. ST_AsTIFF(raster, "JPEG60") (copied from http://www.gdal.org/frmt_gtiff.html)
Open Question: What if we want to export only the first two band of a three band layer?
Maybe we need a ST_RasterFromBands(band1, band2, etc…) to reconstitute a multiband raster from multiple sources (having the same width, height, pixelsize, etc…)
mloskot: or ST_RasterFromBands(bands) where bands is ARRAY[int]. For instance, ST_RasterFromBands(ARRAY[1,3]) will burn new raster from 1 and 3 bands of input raster.
ST_AsPNG(raster, band) → PNG as "bytea"
ST_AsGDALRaster(raster, band int, type text, options text) → bytea
Use GDAL to convert the raster into one of the format suported by GDAL.
Objective FV.02 - Being able to intersect vector and raster to produce raster.
ST_Intersects(raster, raster)
ST_AsRaster(geometry, pixelsize) → raster
ST_Intersection(geometry, val, raster, band) → raster
The first series of variant return a raster having the same extent as the provided raster.
Variant 1: ST_Intersection(geometry, val, raster, band, pixeltype, nodatavalue) → raster
Variant 2: ST_Intersection(raster, band, geometry, val, pixeltype, nodatavalue) → raster
Variant 3: ST_Intersection(geometry, val, raster, pixeltype, nodatavalue) → raster
Variant 4: ST_Intersection(raster, geometry, val, pixeltype, nodatavalue) → raster
The second series of variant return a raster having the minimal extent.
Variant 5: ST_Intersection(geometry, val, raster, band, pixeltype, nodatavalue, 'TRIM') → raster
Variant 6: ST_Intersection(raster, band, geometry, val, pixeltype, nodatavalue, 'TRIM') → raster
Variant 7: ST_Intersection(geometry, val, raster, pixeltype, nodatavalue, 'TRIM') → raster
Variant 8: ST_Intersection(raster, geometry, val, pixeltype, nodatavalue, 'TRIM') → raster
Returns a two bands raster the first band containing only the pixels from the provided raster intersecting with the geometry and the second band containing the same area filled with the provided value.
The second band gets its pixeltype and nodatavalue from the parameters.
Non intersecting pixels are filled with nodata values.
Variant 1 return a raster having the same extent as the provided raster.
Variant 3, 4, 7 and 8 defaults the band number to 1.
Variant 5 to 8 "trim" or "crop" the raster to the withvalue extent (removing extra nodata value pixels surrounding the extent of the resulting withvalue extent).
Open question
PR: Shoud we return one raster per raster/geometry couple or split the raster into as many small rasters as there are areas sharing a same value? The second behavior seems more coherent with the present behavior of ST_Intersection(raster, geometry) → geometry even if this would produce tons of small two bands rasters.
Implementation details
Rasterize the geometry as a new raster (ST_AsRaster(geometry, pixeltype, val, nodataval, raster)) and then copy only pixels for which both raster bands have a value. Should be implemented as a wrapper around ST_MapAlgebra after rasterizing the geometry to a raster having the same alignment as the raster.
Objective FV.03 - Being able to use neighbour pixels in MapAlgebra expressions.
For now ST_MapAlgebra expressions refer only to the pixel being computed. e.g. "rast * 100". The original plan was to allow refering to neighbour pixels using two coordinated relative to the pixel being computed. e.g. "rast[-1,0] * 100" where rast[-1,0] refer to the value of the pixel one pixel to the left of the pixel being computed. However this syntax might prove to be hard to use when many neighbours are to be used.
An alternative syntax would involve another function name (e.g. ST_MapAlgebraNgb) and a way to define a neighbour rectangular region around the computed pixel (e.g.: "2,2" meaning a rectangle encompassing the two neighbour pixels in each direction) and a function to call with this matrix of pixel values. A complete example might look like:
SELECT ST_MapAlgebraNgb(rast, "ST_Mean", 2, 2, "ignore")
So this would mean "for each pixel, compute the average of all the 1 + 8 + 16 = 25 pixels surrounding the current pixel and "ignore" pixels with nodata values."
The "ST_Mean" summarizing function should accept three parameters: an array of float8 values, a X and a Y dimension, and optionnally a "what to do with nodata values". The possible value for this last parameter could be:
- "NULL": If any value is a nodata value, return NULL.
- "ignore": Ignore any nodata value so that if 4 pixels on 25 are nodata values, do the computation with the 21 remaining. This is the default.
- "value": Replace any nodata value with the value of the pixel being computed.
- a value: Replace any nodata value with this value and compute.
Any remaining parameters to ST_MapAlgebraNgb could be passed to the summarizing functions for its own need (e.g. "round" to specify that only the pixel forming a circle should be used in the computing).
A number of predefined summarizing function could be delivered: ST_Max, ST_Min, ST_Sum, ST_Distinct, ST_Mean, ST_STD, ST_Range, ST_Quantile, ST_Median, ST_Majority, ST_Minority, ST_Slope, ST_Aspect, and more…
Users could write their own map algebra summarizing functions.
A more sophisticated version would pass a georeferenced raster instead of just a value matrix so that summarizing function could use this geoereference (e.g. to determine a value from the whole coverage (with ST_Value) when the neighbours are out of the bound of the raster). Passing a raster would allow existing raster functions (like the summarizing function which are to come) and remove the need to pass the dimension since they would correspond to the ST_Width & ST_Height functions. Only the optional "what to do with nodata values" could be needed and some additional parameters.
Three difficulties must be solved to implement this function:
- The matrix values assigment must be optimized when passing from one pixel to the other.
- We must see how it is possible to call a PL/pgSQL function from a C function
- We must see how to pass a variable number of parameter to a PL/pgSQL function
Objective FV.04 - Being able to use "group by" to accumulate tiles to form a new raster.
ST_Union(raster|geometry, raster|geometry, ‘raster’|’geometry’) → raster/geometry
ST_Accum(raster set|geometry set, ‘raster’|’geometry’) → raster/geometry
Objective FV.05 - Being able to reproject a raster.
ST_Transform(raster|geometry, SRID) → same type as input
Objective FV.06 - Being able to do some base raster operations.
ST_Area(raster|geometry) → double
ST_Count(raster, value) → integer
ST_Resample(raster, method, originx, originy, pixelsizex, pixelsizey) → raster
Variant
1) ST_Resample(raster, method, raster) → raster
Recompute a raster in order to change its pixel size and/or the position of its upper left corner.
The second parameter is the resampling method performed when computing new pixel values:
1) 'NEAREST NEIGHBOR'
2) 'LINEAR'
3) 'BICUBIC'
The 3rd or 3rd-6th parameters define the pixel size of the resampling operation. For the 3rd parameter variant, the pixel size is taken from the pixel size of the raster parameter. For the 3rd-6th parameter variant, the pixel origin and dimensions are explicitly defined.
ST_SelectByValue(raster|geometry, ‘expression’) → same type as first argument
ST_Reclass(raster|geometry,string) → same type as first argument
ST_MapAlgebra(raster|geometry, [raster|geometry,…] ‘mathematical expression’, ‘raster’ |’geometry’) → raster/geometry
Variant 1: ST_MapAlgebra(raster|geometry, [raster|geometry,…] ‘mathematical expression’, ‘raster’ |’geometry’, originx, originy, pixelsizex, pixelsizey) → raster/geometry
Variant 2: ST_MapAlgebra(raster|geometry, [raster|geometry,…] 'mathematical expression', 'raster' |'geometry', integer) →raster/geometry
Variant 3: ST_MapAlgebra(raster|geometry, [raster|geometry,…] 'mathematical expression', 'raster' |'geometry', raster) →raster/geometry
ST_MapAlgebra performs the specified mathematical expression on the input rasters, returning a raster or geometry. Both rasters must have the same SRID. If both rasters do not have the same SRID, ST_MapAlgebra will return an error:
ERROR: Operation on two geometries with different SRIDs
The first raster passed to ST_MapAlgebra is the 'master' raster, unless either:
1 additional parameter specifies the index (in the parameter list) of the 'master' raster.
1 additional parameter specifies a raster whose origin and cell size should be used to compute the output raster.
4 additional parameters are passed specifying the origin, cell size, and raster rotation.
This function implicitly calls ST_Intersects(raster|geometry, [raster|geometry,…]) to detect if the rasters are overlapping before performing any computation. Additionally, the resulting raster will have the same extent as the result of ST_Intersection(raster, integer, geometry).
One of the implications of the ST_Intersects inclusion is that:
SELECT ST_MapAlgebra(rast1, rast2, mathExpr) FROM mytable WHERE ST_Intersects(rast1, rast2)
will be faster than:
SELECT ST_MapAlgebra(rast1, rast2, mathExpr) FROM mytable
Open Question:
DZ: Should ST_MapAlgebra resample rasters internally, or produce a raster that can be processed by ST_Resample? If so, variant 1 and variant 3 can be removed, and all ST_MapAlgebra results can be processed through ST_Resample, like:
ST_Resample(ST_MapAlgebra(raster, [raster,…] 'mathematical expression', integer), originx, originy, pixelsizex, pixelsizey)
ST_Resample(ST_MapAlgebra(raster, [raster,…] 'mathematical expression', integer), rastergrid)PR: I think this would greatly contribute to simplify the API.
ST_Clip(raster|geometry,geometry) → same type as first argument
ST_Flip(raster|geometry, ’vertical’|’horizontal’) → same type as first argument
Objective FV.07 - Being able to convert a raster to standards formats.
ST_AsKML(raster|geometry) → string
ST_AsSVG(raster|geometry) → string
Objective FV.08 - Being able to control the validity of a raster.
ST_IsEmpty(raster|geometry) → boolean
ST_mem_size(raster|geometry) → integer
ST_isvalid(raster|geometry) → boolean
Objective FV.09 - Being able to use other major topological operators
ST_Within(raster|geometry A, raster|geometry B)
ST_Contains(raster|geometry A, raster|geometry B)
ST_Overlaps(raster|geometry, raster|geometry)
Objective FV.10 - Being able to derive a raster layer from vector layer.
ST_Interpolate(points, pixelsize, method) → raster
Objective FV.11 - Being able to do on rasters most operations available on geometries
ST_Centroid(raster|geometry) → point geometry
ST_PointOnSurface(raster|geometry) → point geometry
ST_Buffer(raster|geometry, double) → same type as first arg.
ST_Difference(raster|geometry A, raster|geometry B) → same type as first argument
ST_SymDifference(raster|geometry,raster|geometry,‘raster’|’geometry’) → raster/geometry
Objective FV.12 - Being able to use all the other topological operators
ST_Equals(raster|geometry, raster|geometry)
ST_Disjoint(raster|geometry, raster|geometry)
ST_Touches(raster|geometry, raster|geometry)
ST_Crosses(raster|geometry, raster|geometry)
ST_Covers(raster|geometry A, raster|geometry B)
ST_IsCoveredBy(raster|geometry A, raster|geometry B)
ST_Relate(raster|geometry, raster|geometry, intersectionPatternMatrix )
Objective FV.13 - Being able to edit a raster
ST_Affine(raster|geometry,…) → same type as input
ST_Translate(raster|geometry,…) → same type as input
ST_Scale(raster|geometry,…) → same type as input
ST_TransScale(raster|geometry,…) → same type as input
ST_RotateZ,Y,Z(raster|geometry, float8) → same type as input
Objective FV.14 - Being able to intersect two rasters to get a raster.
ST_Intersection(raster, integer, raster, integer) → raster - Returns a two bands raster with values only in the intersecting areas of both rasters. Integer parameters are the band number of the raster.
Variants
1) ST_Intersection(raster, integer, raster, integer) → raster — the integer parameters are the band number of the rasters
2) ST_Intersection(raster, raster, integer) → raster — default first raster to band # 1
3) ST_Intersection(raster, integer, raster) → raster — default second raster to band # 1
4) ST_Intersection(raster, raster) → raster — default both rasters to band # 1
Objective FV.15 - Being able to intersect two rasters to get a geometry.
ST_Intersection(raster, integer, raster, integer, 'geometry') → geometry - Returns a two bands raster with values only in the intersecting areas of both rasters. Integer parameters are the band number of the raster.
Variants
1) ST_Intersection(raster, integer, raster, integer, 'geometry') → geometry
2) ST_Intersection(raster, raster, integer, 'geometry') → geometry — default first raster to band # 1
3) ST_Intersection(raster, integer, raster, 'geometry') → geometry — default second raster to band # 1
4) ST_Intersection(raster, raster, 'geometry') → geometry — default both raster to band # 1
Objective FV.16 - Being able to quickly get raster statistics.
Add cached basic raster statistic to the base raster WKB format.
Objective FV.17 - Being able to refer to band by textual name.
Add 8 digit string to each band in the base raster WKB format.
Adjust gdal2wktraster.py to be able to give names to each band when importing.
Adjust/overlaod every function to be able to refer to raster band by name.
Objective FV.18 - Being able to load rasters from SQL
The idea is to change the rt_band_get_data core function so it load filesystem registered raster data using GDAL into the data base. This allow us to create a list of raster with a new ST_MakeRegisteredRaster("c:/temp/mytiff/*.tif") and to convert them witinot a CREATE TABLE with a ST_MakeBandInDB(rast, band)
Changes to the rt_band_get_data core function
ST_MakeRegisteredRaster(wildcardPath)
ST_SetPath(raster, band, string)
ST_MakeBandInDB(rast, band)
Other functions
ST_AsBinary(raster, compression)
ST_RasterFromWKB(raster, [<srid>])
ST_RasterFromText(string, [<srid>])
ST_AsText(raster)
