| Version 3 (modified by , 13 years ago) ( diff ) |
|---|
GDAL Driver for PostGIS Raster Working Specifications
Quick Links
Current status of the driver (February 2011)
The driver is:
- Able to read in-db evenly blocked rasters (all blocks with same size)
- Able to read in-db one-row-rasters:
- If the table really has more than one row: using -where clause in connection string
- If the table has more than one row: the table must have been marked as "regularly blocked table", with -k in loader
- Able to produce generate two kind of raster object based on two modes:
- ONE_RASTER_PER_ROW ('mode = 1' in connection string, or nothing): The default mode. Each table row is considered as an independent raster. If the table required has more than one row, and no -where clause has been specified in connection string, all the table rows will be considered as reported as Subdatasets. Unless you specify the other working mode
- ONE_RASTER_PER_TABLE ('mode = 2' in connection string): Each table is considered as a raster coverage, and each row is a raster tile.
- Too slow (Reads the entire table metadata for constructing the GDALDataset object, and needs one server round per GDALRasterBand::IReadBlock call)
The driver is not:
- Able to read out-db rasters (developed, but not tested, and with known bugs)
- Able to create new rasters
- Able to manage all the PostGIS Raster arrangements
- Able to provide a color interpretation for bands
Design principles
Topic: The basis
The main class of a GDAL driver is GDALDataset: A set of associated raster bands. So, 1 GDALDataset must be able to contain:
- An untiled image stored in a raster table's row.
- A tiled image stored in a raster table (regular or irregular, rectangular or not, with or without missing tiles, with or without overlapping between tiles)
- A raster object coverage from the rasterization of a vector coverage stored in a raster table (regular or irregular, rectangular or not, with or without missing tiles, with or without overlapping between tiles)
In the first case, 1 GDALDataset = 1 PostGIS Raster object. In the other two cases, 1 GDALDataset = Several PostGIS Raster objects. For this reason, the GDAL PostGIS Raster driver has 2 working modes: ONE_RASTER_PER_TABLE, ONE_RASTER_PER_ROW.
However, currently the driver only deals with continuous tiled raster layers, when all the raster tiles are the same size, snap to the same grid and do not overlap (the ideal case).
Open question: Are 2 working modes enough to manage all the raster arrangements?
Topic: Constructing the GDALDataset object
To construct a GDALDataset object, the driver must:
- Open the dataset (create db connection)
- Read some data about the dataset (metadata): srid, georeference information, projection information, raster data size, band information (number of bands, pixel size, color interpretation, if present), any other driver-specific dataset related information (i.e.: in our case, schema and table name)
- Construct the structure for raster bands, with instances of GDALRasterBand class. You need to provide some basic information: data type (pixel size), block size (GDAL contains a concept of the natural block size of rasters so that applications can organized data access efficiently for some file formats) and color interpretation (if any).
The metadata must be read from the raster table, using SQL functions like ST_Extent (used for raster data extent), ST_Metadata (used for general raster metadata) or functions like ST_SRID, ST_Width, ST_Height, etc. When your GDALDataset matches only one raster row (a raster tile) this is not a problem. But when your GDALDataset matches a whole raster table (ONE_RASTER_PER_TABLE mode), you have 2 options:
- Call the functions over the whole table and filter the result (i.e.: select distinct st_srid(rast) from raster_table, select distinct st_metadata(rast) from raster table). It can be a really slow operation, but you can check if all tiles are like expected (for example: if they are the same size, if they share the same srid, if they overlap or not, etc)
- Call the functions limiting the output to one result. Fast operation, but may be incorrect
Currently, the driver takes the first (and slow) option. That caused performance problems (see ticket #497)
Open question: How to fetch the information needed to construct the GDALDataset? Pay attention to the fact that you are not asking for raster data yet. You only need metadata, for constructing the basic GDALDataset object.
Topic: Reading/Writing raster data
Once constructed the basic structure (GDALDataset object and related GDALRasterBand objects), you need to choose the strategy for raster data reading/writing:
- Natural block oriented r/w: The driver reads/writes data in equal sized blocks. The potentially more efficient way of r/w data. Really, the natural block size for this dataset is chosen during GDALRasterBand creation. So, it's driver's responsibility to provide the desired value for block size. To use this method, your driver must provide an implementation of IReadBlock.
- Region oriented r/w: The driver reads/writes arbitrary regions of data. It's a potentially less efficient method, because you have to take care of data type translation if the data type of the buffer is different than that of the GDALRasterBand. You also must takes care of image decimation / replication if the buffer size (nBufXSize x nBufYSize) is different than the size of the region being accessed (nXSize x nYSize). To use this method, your driver must provide an implementation of IRasterIO.
Clearly, there's no best method for reading/writing data in our case. In the ideal case of regulary blocked rasters, with no overlapping and same grid for all tiles, the block oriented r/w is the more appropiate strategy. But in the rest of the cases, a more general r/w method must be provided.
Currently, the natural block oriented r/w method is the one implemented for the driver. This is a limitation for 2 reasons:
- Obviously, it only fits one raster arrangement
- Each ReadBlock call forces a new server round, constructing a Box and getting the raster row that contains it. This can be really slow, in case of huge raster coverages (question raised in ticket #497 too).
Open question: How to get the needed metadata in case of ONE_RASTER_PER_TABLE arrangement. As argued in ticket #497, executing ST_Extent or ST_Metadata without limits over a big table can be a really heavy process.
Open question: What should be the general r/w algorithm? jorgearevalo: I think the strategy read as much data as you can should be the right one, to minimize server rounds. This is: construct a query that, using ST_Intersects, fetches as much rows as possible. This query would be executed in IRasterIO method. But I don't know how to choose the geographic limits for the query (how much data is as much data as you can?)
