= '''PostGIS Raster Working Specifications for Future Versions''' = {{{ #!div style='background-color: #F4F4F4; padding: 5px; border: 1px solid gray; float: right; margin-left: 5px; width: 260px; font-size: small;' > '''Quick Links''' * [wiki:WKTRaster PostGIS Raster Home Page] * [wiki:WKTRaster/PlanningAndFunding Planning & Funding] * [wiki:WKTRaster/SpecificationWorking02 Working Specifications for PostGIS 2.0] * [wiki:WKTRaster/SpecificationFinal01 Old Final Specifications for Beta 0.1.6] }}} ---- == '''Objective FV.01 - Being able to return a JPEG, a TIFF, a PNG or any image format supported by GDAL.''' == '''ST_bytea(raster, integer) -> raster''' -- the integer parameters is the band number of the raster.[[BR]] What is does? ---- '''Open Question:''' When exporting a multiband raster to JPEG, TIFF, PNG, SVG or KML, how should we specify the band number in the exporting function. There is two options to select the band to convert from a multiband raster in all the ST_AsFormat functions. [[BR]] [[BR]] 1. Precede each call with ST_Band() to return a selected band.[[BR]] Pros: This is a general function that can be called before any function that would otherwise require a band parameter.[[BR]] Cons: This implies creating a temporary raster. This might be more elegant and general but is this too much overhead comparing with having a band parameter? 2. Add a band parameter to each ST_AsFormat function.[[BR]] Pros: Hypothetically less overhead.[[BR]] Cons: Every functions implying access to a band should then have this parameter when in most case it would be equal to 1. In many cases it makes no sence to have to specify a band parameter since it is the whole raster that we want to export, including all the bands. Pierre: More I think about it more I think that the first option is the best one... mloskot: Perhaps there is a compromise in form of two sets of functions: 1) ST_As* which always burn the whole raster (all bands) 2) ST_BandAs* which takes number of band as a parameter and return only this requested band. ---- '''ST_Band(raster, integer) -> raster''' -- the integer parameters are the band number of the rasters.[[BR]] Return a single band from a multiband raster. If "band" is greater than the value returned by ST_GetNumBands(), the function returns the last band. This function should be used to select a band before converting it to JPEG, TIFF, PNG, SVG or KML with the corresponding function. e.g. '''ST_AsTIFF(ST_Band(raster, band)) ---- '''ST_AsJPEG(raster, quality) -> JPEG as "bytea"'''[[BR]] Return the raster as a JPEG encoded as a PostgreSQL bytea. By default quality is set to 75, but this option can be used to select other values. Values must be in the range 10-100. Low values result in higher compression ratios, but poorer image quality. Values above 95 are not meaningfully better quality but can but substantially larger. (copied from http://www.gdal.org/frmt_jpeg.html) '''Open Question:''' Is JPEG export limited to raster having 8 bit unsigned integer pixeltype (8BUI)? [http://www.gdal.org/frmt_jpeg.html See how GDAL do it]. It converts only 8 bits rasters. Should we do the same? Otherwise, how do we convert other types to 8BUI? e.g. 16BUI or 8BSI? Pierre: It might be more simple to ignore pixeltypes other than 8BUI but it would be very convenient to have a way to quickly export elevation data for example as a JPEG. It would be nice to have an elegant solution to this. Maybe something inspired from !MapServer. Proposition one (Pierre): ST_AsJPEG could simply (optionally when the pixeltype is not 8BUI) map the ST_Maximum() and ST_Minimum() value to 0-255. ST_Maximum() and ST_Minimum() are not in the spec yet but this could be on nice usage of it. They will imply caching the min and max when importing and editing. Both function should ignore the !NoDataValues. They could also be two parameters passed to ST_AsJPEG(raster, quality, min, max). Proposition two: There could also be just one parameter (string) defining a mapping method: * Method "None": No mapping. This is possible only for 8BUI. * Method "!MaxMinValue": Use the Max and the Min cached in the raster. e.g. for 16BSI (min, max) -> (-2033, 2456) -> (round((-2033 - -2033)/(2456 - -2033)*255), round((2456 - -2033)/(2456 - -2033)*255)) -> (0, 255).[[BR]] [[BR]]This is equivalent to ST_AsJPEG(raster, quality, ST_Minimum(rast), ST_Maximum(rast)) * Method "!MaxMinType": Use the Max and the Min allowed by the type. e.g. for 16BSI (min, max) -> (-2033, 2456) -> (round((-2033 - -32768)/(32767 - -32768)*255), round((2456 - -32768)/(32767 - -32768)*255)) -> (120, 137)[[BR]] [[BR]]This would be equivalent to ST_AsJPEG(raster, quality, ST_BandPixelTypeMin(rast), ST_BandPixelTypeMax(rast)). Both functions (ST_BandPixelTypeMin & ST_BandPixelTypeMax) are not yet planned and I could not find an SQL query that returns the equivalent range for a type. [http://groups.google.nl/group/microsoft.public.sqlserver.programming/browse_thread/thread/46512c2691da4607/6743f4aea485c6d1 One possible solution.] mloskot: ATM, I have no thoughts on this issue. '''Open Question:''' Is JPEG export limited to raster having 1 or 3 bands? [http://www.gdal.org/frmt_jpeg.html See how GDAL do it]. It converts only 1 or 3 band rasters. Should we do the same? In this case 1 band rasters would be exported as a greyscale JPEG having R G and B identical and 3 band rasters would be interpreted as R, G and B. Pierre: I think the answer should be yes. I don't see how we could have a 2 band raster fit into RGB. mloskot: I agree, the answer should be yes. '''Here is an attempt to define the different versions of the function:''' The most minimalistic versions of the function should assume band 1, 2 and 3 as being r, g, b and the quality equal to 75: ST_AsJPEG(raster) -quality = 75 A variant allow specifying the quality: ST_AsJPEG(raster, integer) Another variant should enable us to specify which band correspond to the r, the g and the b: ST_AsJPEG(raster, integer, integer, integer) - raster, rband, gband, bband, quality=75 ST_AsJPEG(raster, integer, integer, integer, integer) - raster, rband, gband, bband, quality Another version should be designed to be used with a future ST_Band(raster) function. In this case there is no attempt to extract r, g or b band from any passed raster: ST_AsJPEG(raster, raster, raster) ST_AsJPEG(raster, raster, raster, integer) -with the quality param Another series should allow converting 1 band raster with pixel of type 8BUI to a grayscale JPEG (Carefull study of the GDAL behavior when converting a single band to JPEG should be done before confirming these functions): ST_AsJPEG(raster, "GRAYSCALE") - convert only band 1 with quality = 75 ST_AsJPEG(raster, "GRAYSCALE", integer) - convert only band 1 with specified quality ST_AsJPEG(raster, integer, "GRAYSCALE") - allow specifying the band number to convert ST_AsJPEG(raster, integer, "GRAYSCALE", integer) - allow specifying the band number to convert and the quality Another series should allow converting 1 band raster of ANY pixel type to a grayscale JPEG. Pixel types different than 8BUI should be mapped according to specified min, max values and a mapping mode: "!MaxMinValue" (default) or "!MaxMinType". ST_AsJPEG(raster, "GRAYSCALE", min, max, text) - convert only band 1 with quality = 75 ST_AsJPEG(raster, "GRAYSCALE", integer, min, max, text) - convert only band 1 with specified quality ST_AsJPEG(raster, integer, "GRAYSCALE", min, max, text) - allow specifying the band number to convert ST_AsJPEG(raster, integer, "GRAYSCALE", integer, min, max, text) - allow specifying the band number to convert and the quality ---- '''ST_AsTIFF(raster, compression) -> TIFF as "bytea"'''[[BR]] Return the raster as a JPEG encoded as a PostgreSQL bytea. If raster is a multiband raster and no band were selected with ST_Band() every band are written to the resulting TIFF. compression=[JPEG/LZW/PACKBITS/DEFLATE/CCITTRLE/CCITTFAX3/CCITTFAX4/NONE]: Set the type of compression to use. None is the default. The CCITT compression should only be used with 1bit (NBITS=1) data. JPEG should only be used with Byte data. When using JPEG add a number specifying the quality. 75 is the default. e.g. ST_AsTIFF(raster, "JPEG60") (copied from http://www.gdal.org/frmt_gtiff.html) ---- '''Open Question:''' What if we want to export only the first two band of a three band layer? Maybe we need a ST_RasterFromBands(band1, band2, etc...) to reconstitute a multiband raster from multiple sources (having the same width, height, pixelsize, etc...) mloskot: or ST_RasterFromBands(bands) where bands is ARRAY[int]. For instance, ST_RasterFromBands(ARRAY[1,3]) will burn new raster from 1 and 3 bands of input raster. ---- '''ST_AsPNG(raster, band) -> PNG as "bytea"''' ---- '''ST_AsGDALRaster(raster, band int, type text, options text) -> bytea''' Use GDAL to convert the raster into one of the format suported by GDAL. ---- == '''Objective FV.02 - Being able to intersect vector and raster to produce raster.''' == '''ST_Intersects(raster, raster)'''[[BR]] '''ST_AsRaster(geometry, pixelsize) -> raster'''[[BR]] '''ST_Intersection(geometry, val, raster, band) -> raster''' The first series of variant return a raster having the same extent as the provided raster. Variant 1: ST_Intersection(geometry, val, raster, band, pixeltype, nodatavalue) -> raster Variant 2: ST_Intersection(raster, band, geometry, val, pixeltype, nodatavalue) -> raster Variant 3: ST_Intersection(geometry, val, raster, pixeltype, nodatavalue) -> raster Variant 4: ST_Intersection(raster, geometry, val, pixeltype, nodatavalue) -> raster The second series of variant return a raster having the minimal extent. Variant 5: ST_Intersection(geometry, val, raster, band, pixeltype, nodatavalue, 'TRIM') -> raster Variant 6: ST_Intersection(raster, band, geometry, val, pixeltype, nodatavalue, 'TRIM') -> raster Variant 7: ST_Intersection(geometry, val, raster, pixeltype, nodatavalue, 'TRIM') -> raster Variant 8: ST_Intersection(raster, geometry, val, pixeltype, nodatavalue, 'TRIM') -> raster Returns a two bands raster the first band containing only the pixels from the provided raster intersecting with the geometry and the second band containing the same area filled with the provided value. The second band gets its pixeltype and nodatavalue from the parameters. Non intersecting pixels are filled with nodata values. Variant 1 return a raster having the same extent as the provided raster. Variant 3, 4, 7 and 8 defaults the band number to 1. Variant 5 to 8 "trim" or "crop" the raster to the withvalue extent (removing extra nodata value pixels surrounding the extent of the resulting withvalue extent). '''Open question''' PR: Shoud we return one raster per raster/geometry couple or split the raster into as many small rasters as there are areas sharing a same value? The second behavior seems more coherent with the present behavior of ST_Intersection(raster, geometry) -> geometry even if this would produce tons of small two bands rasters. '''Implementation details''' Rasterize the geometry as a new raster (ST_AsRaster(geometry, pixeltype, val, nodataval, raster)) and then copy only pixels for which both raster bands have a value. Should be implemented as a wrapper around ST_MapAlgebra after rasterizing the geometry to a raster having the same alignment as the raster. ---- == '''Objective FV.03 - Being able to use neighbour pixels in !MapAlgebra expressions.''' == For now ST_MapAlgebra expressions refer only to the pixel being computed. e.g. "rast * 100". The original plan was to allow refering to neighbour pixels using two coordinated relative to the pixel being computed. e.g. "rast[-1,0] * 100" where rast[-1,0] refer to the value of the pixel one pixel to the left of the pixel being computed. However this syntax might prove to be hard to use when many neighbours are to be used. An alternative syntax would involve another function name (e.g. ST_MapAlgebraNgb) and a way to define a neighbour rectangular region around the computed pixel (e.g.: "2,2" meaning a rectangle encompassing the two neighbour pixels in each direction) and a function to call with this matrix of pixel values. A complete example might look like: SELECT ST_MapAlgebraNgb(rast, "ST_Avg", 2, 2, "ignore") So this would mean "for each pixel, compute the average of all the 1 + 8 + 16 = 25 pixels surrounding the current pixel and "ignore" pixels with nodata values." The "ST_Avg" summarizing function should accept three parameters: an array of float8 values, a X and a Y dimension, and optionnally a "what to do with nodata values". The possible value for this last parameter could be: * "NULL": If any value is a nodata value, return NULL. * "ignore": Ignore any nodata value so that if 4 pixels on 25 are nodata values, do the computation with the 21 remaining. * a value: Replace any nodata value with this value and compute. Any remaining parameters to ST_MapAlgebraNgb could be passed to the summarizing functions for its own need (e.g. "round" to specify that only the pixel forming a circle should be used in the computing). A number of predefined summarizing function could be delivered: ST_Average, ST_Majority, ST_Slope, ST_Aspect, Users could write their own map algebra summarizing functions. A more sophisticated version would pass a georeferenced raster instead of just a value matrix so that summarizing function could use this geoereference (e.g. to determine a value from the whole coverage (with ST_Value) when the neighbours are out of the bound of the raster). Passing a raster would allow existing raster functions (like the summarizing function which are to come) and remove the need to pass the dimension since they would correspond to the ST_Width & ST_Height functions. Only the optional "what to do with nodata values" could be needed and some additional parameters. Three difficulties must be solved to implement this function: * The matrix values assigment must be optimized when passing from one pixel to the other. * We must see how it is possible to call a PL/pgSQL function from a C function * We must see how to pass a variable number of parameter to a PL/pgSQL function ---- == '''Objective FV.04 - Being able to use "group by" to accumulate tiles to form a new raster.''' == '''ST_Union(raster|geometry, raster|geometry, ‘raster’|’geometry’) -> raster/geometry'''[[BR]] '''ST_Accum(raster set|geometry set, ‘raster’|’geometry’) -> raster/geometry''' ---- == '''Objective FV.05 - Being able to reproject a raster.''' == '''ST_Transform(raster|geometry, SRID) -> same type as input''' ---- == '''Objective FV.06 - Being able to do some base raster operations.''' == '''ST_Area(raster|geometry) -> double'''[[BR]] '''ST_Count(raster, value) -> integer'''[[BR]] '''ST_Resample(raster, method, originx, originy, pixelsizex, pixelsizey) -> raster''' '''Variant''' 1) ST_Resample(raster, method, raster) -> raster Recompute a raster in order to change its pixel size and/or the position of its upper left corner. The second parameter is the resampling method performed when computing new pixel values: 1) 'NEAREST NEIGHBOR'[[BR]] 2) 'LINEAR'[[BR]] 3) 'BICUBIC' The 3rd or 3rd-6th parameters define the pixel size of the resampling operation. For the 3rd parameter variant, the pixel size is taken from the pixel size of the raster parameter. For the 3rd-6th parameter variant, the pixel origin and dimensions are explicitly defined. '''ST_SelectByValue(raster|geometry, ‘expression’) -> same type as first argument'''[[BR]] '''ST_Reclass(raster|geometry,string) -> same type as first argument'''[[BR]] '''ST_MapAlgebra(raster|geometry, [raster|geometry,…] ‘mathematical expression’, ‘raster’ |’geometry’) -> raster/geometry''' Variant 1: ST_MapAlgebra(raster|geometry, [raster|geometry,…] ‘mathematical expression’, ‘raster’ |’geometry’, originx, originy, pixelsizex, pixelsizey) -> raster/geometry Variant 2: ST_MapAlgebra(raster|geometry, [raster|geometry,…] 'mathematical expression', 'raster' |'geometry', integer) ->raster/geometry Variant 3: ST_MapAlgebra(raster|geometry, [raster|geometry,…] 'mathematical expression', 'raster' |'geometry', raster) ->raster/geometry ST_MapAlgebra performs the specified mathematical expression on the input rasters, returning a raster or geometry. Both rasters must have the same SRID. If both rasters do not have the same SRID, ST_MapAlgebra will return an error: ERROR: Operation on two geometries with different SRIDs The first raster passed to ST_MapAlgebra is the 'master' raster, unless either: 1 additional parameter specifies the index (in the parameter list) of the 'master' raster.[[BR]] 1 additional parameter specifies a raster whose origin and cell size should be used to compute the output raster.[[BR]] 4 additional parameters are passed specifying the origin, cell size, and raster rotation. This function implicitly calls ST_Intersects(raster|geometry, [raster|geometry,…]) to detect if the rasters are overlapping before performing any computation. Additionally, the resulting raster will have the same extent as the result of ST_Intersection(raster, integer, geometry). One of the implications of the ST_Intersects inclusion is that: SELECT ST_MapAlgebra(rast1, rast2, mathExpr) FROM mytable WHERE ST_Intersects(rast1, rast2) will be faster than: SELECT ST_MapAlgebra(rast1, rast2, mathExpr) FROM mytable '''Open Question:''' DZ: Should ST_MapAlgebra resample rasters internally, or produce a raster that can be processed by ST_Resample? If so, variant 1 and variant 3 can be removed, and all ST_MapAlgebra results can be processed through ST_Resample, like: ST_Resample(ST_MapAlgebra(raster, [raster,…] 'mathematical expression', integer), originx, originy, pixelsizex, pixelsizey)[[BR]] ST_Resample(ST_MapAlgebra(raster, [raster,…] 'mathematical expression', integer), rastergrid) PR: I think this would greatly contribute to simplify the API. '''ST_Clip(raster|geometry,geometry) -> same type as first argument'''[[BR]] '''ST_Flip(raster|geometry, ’vertical’|’horizontal’) -> same type as first argument''' ---- == '''Objective FV.07 - Being able to convert a raster to standards formats.''' == '''ST_AsKML(raster|geometry) -> string'''[[BR]] '''ST_AsSVG(raster|geometry) -> string''' ---- == '''Objective FV.08 - Being able to control the validity of a raster.''' == '''ST_IsEmpty(raster|geometry) -> boolean'''[[BR]] '''ST_mem_size(raster|geometry) -> integer'''[[BR]] '''ST_isvalid(raster|geometry) -> boolean''' ---- == '''Objective FV.09 - Being able to use other major topological operators''' == '''ST_Within(raster|geometry A, raster|geometry B)'''[[BR]] '''ST_Contains(raster|geometry A, raster|geometry B)'''[[BR]] '''ST_Overlaps(raster|geometry, raster|geometry)''' ---- == '''Objective FV.10 - Being able to derive a raster layer from vector layer.''' == '''ST_Interpolate(points, pixelsize, method) -> raster''' ---- == '''Objective FV.11 - Being able to do on rasters most operations available on geometries''' == '''ST_Centroid(raster|geometry) -> point geometry'''[[BR]] '''ST_PointOnSurface(raster|geometry) -> point geometry'''[[BR]] '''ST_Buffer(raster|geometry, double) -> same type as first arg.'''[[BR]] '''ST_Difference(raster|geometry A, raster|geometry B) -> same type as first argument'''[[BR]] '''ST_SymDifference(raster|geometry,raster|geometry,‘raster’|’geometry’) -> raster/geometry''' ---- == '''Objective FV.12 - Being able to use all the other topological operators''' == '''ST_Equals(raster|geometry, raster|geometry)'''[[BR]] '''ST_Disjoint(raster|geometry, raster|geometry)'''[[BR]] '''ST_Touches(raster|geometry, raster|geometry)'''[[BR]] '''ST_Crosses(raster|geometry, raster|geometry)'''[[BR]] '''ST_Covers(raster|geometry A, raster|geometry B)'''[[BR]] '''ST_IsCoveredBy(raster|geometry A, raster|geometry B)'''[[BR]] '''ST_Relate(raster|geometry, raster|geometry, intersectionPatternMatrix )''' ---- == '''Objective FV.13 - Being able to edit a raster''' == '''ST_Affine(raster|geometry,…) -> same type as input'''[[BR]] '''ST_Translate(raster|geometry,…) -> same type as input'''[[BR]] '''ST_Scale(raster|geometry,…) -> same type as input'''[[BR]] '''ST_TransScale(raster|geometry,…) -> same type as input'''[[BR]] '''ST_RotateZ,Y,Z(raster|geometry, float8) -> same type as input''' ---- == '''Objective FV.14 - Being able to intersect two rasters to get a raster.''' == '''ST_Intersection(raster, integer, raster, integer) -> raster''' - Returns a two bands raster with values only in the intersecting areas of both rasters. Integer parameters are the band number of the raster. '''Variants''' 1) ST_Intersection(raster, integer, raster, integer) -> raster -- the integer parameters are the band number of the rasters[[BR]] 2) ST_Intersection(raster, raster, integer) -> raster -- default first raster to band # 1[[BR]] 3) ST_Intersection(raster, integer, raster) -> raster -- default second raster to band # 1[[BR]] 4) ST_Intersection(raster, raster) -> raster -- default both rasters to band # 1 ---- == '''Objective FV.15 - Being able to intersect two rasters to get a geometry.''' == '''ST_Intersection(raster, integer, raster, integer, 'geometry') -> geometry''' - Returns a two bands raster with values only in the intersecting areas of both rasters. Integer parameters are the band number of the raster. '''Variants''' 1) ST_Intersection(raster, integer, raster, integer, 'geometry') -> geometry[[BR]] 2) ST_Intersection(raster, raster, integer, 'geometry') -> geometry -- default first raster to band # 1[[BR]] 3) ST_Intersection(raster, integer, raster, 'geometry') -> geometry -- default second raster to band # 1[[BR]] 4) ST_Intersection(raster, raster, 'geometry') -> geometry -- default both raster to band # 1 ---- == '''Objective FV.16 - Being able to quickly get raster statistics.''' == '''Add cached basic raster statistic to the base raster WKB format. ---- == '''Objective FV.17 - Being able to refer to band by textual name.''' == '''Add 8 digit string to each band in the base raster WKB format. '''Adjust gdal2wktraster.py to be able to give names to each band when importing. '''Adjust/overlaod every function to be able to refer to raster band by name. ---- == '''Objective FV.18 - Being able to load rasters from SQL''' == The idea is to change the rt_band_get_data core function so it load filesystem registered raster data using GDAL into the data base. This allow us to create a list of raster with a new ST_MakeRegisteredRaster("c:/temp/mytiff/*.tif") and to convert them witinot a CREATE TABLE with a ST_MakeBandInDB(rast, band) '''Changes to the rt_band_get_data core function'''[[BR]] '''ST_MakeRegisteredRaster(wildcardPath)'''[[BR]] '''ST_SetPath(raster, band, string)'''[[BR]] '''ST_MakeBandInDB(rast, band)''' ---- == '''Other functions''' == '''ST_AsBinary(raster, compression)'''[[BR]] '''ST_RasterFromWKB(raster, [])'''[[BR]] '''ST_RasterFromText(string, [])'''[[BR]] '''ST_AsText(raster)'''