| 48 | | Then, for the basic version of the GDAL WKT Raster driver, we can focus on these tasks (reading raster of types c, d and e). Anyway, in the project plan, all the mandatory and optional tasks are listed. |
| | 49 | Then, for the basic version of the GDAL WKT Raster driver, we can focus on these tasks (reading raster of types c and d). For this reason, we specify several working modes for it. They aren't totally defined yet, but, at least, we should use: |
| | 50 | - REGULARLY_BLOCKING mode: In this mode, our raster is regularly tiled. So, it fulfills the three previous properties. |
| | 51 | - NON_REGULARLY_BLOCKING mode: In this mode, our raster is non-regularly tiled. The tiles can have different size and may overlap, so, the global extent of the raster isn't necessarily rectangular. |
| | 52 | |
| | 53 | Maybe we'll need more working modes in the future, for non-tiled rasters. |
| 58 | | |
| 59 | | There are three important methods that must be implemented in the Dataset class: |
| 60 | | * Open: a static method that establishes a connection with PostgreSQL and try to access the table given as argument. Several security checkings must be performed here: if the database has PostGIS support, if the table (or tables) exists, if has a field of raster type, if has a GIST index, etc. (Any more?). Additionally, this method will create the RasterBand objects, needed for fetching the raster bands' data, and create a pointer to the real data. |
| 61 | | * GetGeoTransform: Fetches the coefficients for transforming between pixel/line (P,L) raster space and projection coordinates (Xp, Yp) space. |
| 62 | | * GetProjectionRef: Fetches the projection coordinate system definition in OpenGIS WKT format. It should be suitable for use with the OGRSpatialReference class. |
| 63 | | |
| 64 | | This is for Dataset based on regular blocking arrangements. If we have non-regular blocking arrangements, things turn more complicated. And how do we know if the table can be considered as a raster coverage with regular blocking arrangement? By querying the '''raster_columns''' table. |
| 65 | | |
| 66 | | This table is a key concept in the WKT Raster world. ''Like the PostGIS "geometry_column" table, the PostGIS WKT Raster "raster_columns" table is a way for applications to get a quick overview of which tables have a raster column, and to get the main characteristics (metadata) of the rasters stored in these columns.'' |
| 67 | | |
| 68 | | One of this table's fields is a boolean field called 'regular_blocking'. ''If it is set to 'true' then you treat all the rows as normal tiles and create a single GDALDataset. If it is set to 'false' then you should print an error message warning to check for the "MODE" option. This MODE option could work like this:'' |
| 69 | | |
| 70 | | ''-MODE=WHOLE_TABLE, in this case you first query the full extent covered by all the raster contained in the table, create a 'black' image following this extent with nodata values and then 'print' every images in the big raster. This is very similar as if the regular_blocking flag is set but you will have to deal with true georeferences and possible overlaps. This should be very similar to what the gdal_merge.py tool does.'' |
| 71 | | |
| 72 | | ''-MODE=FIRST_ROW, in this case you treat only the first row returned by a SQL query. This could be very similar to the 'where' parameter of the GEORASTER GDAL driver.'' |
| 73 | | |
| 74 | | ''-MODE=ONE_FILE_PER_ROW, in this case you would read each row as an independent image. I'm not sure though that GDAL allows you to do this. For sure a good WKT exporter/dumper would allow this. The result would be one filesystem raster file per row.''. UPDATE: Subdatasets (http://www.gdal.org/gdal_datamodel.html, SUBDATASETS Domain) could be useful here. Can datasets based on a single row treated as tiles (blocks), this is, loaded in RAM, the whole image? Maybe is better to provide finer grained access to this datasets, but it will mean extra work. |
| 75 | | |
| 76 | | **** QUESTION: There is one type of regular-blocking WKT Raster that implies more than one table, the type called "tiled image". In this type, we'd have to read two or more tables to get the whole raster coverage. So, do we have to read all tables, with all of their tiles, to create a Dataset? In this case, the connection string syntax may vary... |
| 77 | | |
| 78 | | **** ANSWER, by Pierre Racine: I would not take care of this arrangement. Lets focus on reading one table at a time. As I said in the doc this arrangement is not very useful/modern. |
| 79 | | |
| 80 | | |
| 81 | | **** QUESTION: How to manage the mode in irregular-blocking rasters? |
| 82 | | |
| 83 | | **** ANSWER: Maybe an additional param to connection string: PG:......mode='<mode>'. Even for regular blocking arrangements too. |
| 84 | | |
| 85 | | Anyway, as we said, I'm going to focus on reading the regular blocking arrangements. And how should I read these arrangements? |
| 86 | | |
| 87 | | I think that I need 2 steps: |
| 88 | | 1. Read the raster attributes using information from RASTER_COLUMNS table. For example: The parameters to GeoTransform operation must be taken from the RASTER_COLUMNS table. Update: done on r20 |
| 89 | | 1. Read the raster data as a HEXWKB stream, allocating enough memory for all the tiles. |
| 90 | | |
| 91 | | **** QUESTION: Should the RasterBand read all the tiles (rows of the raster table) and put them consecutives in the same memory area? If I expect this from RasterBand class, the memory allocation is easy. |
| 92 | | |
| 93 | | **** ANSWER: It should have only one block (tile) in memory at same time. |
| | 61 | 1. Check connection string format |
| | 62 | 1. Parse connection string, extracting useful information (table name, optional sql where part, working mode) |
| | 63 | 1. Check working mode. If not regularly_tiled mode, from now, finish. |
| | 64 | 1. Open a database connection. |
| | 65 | 1. Perform some security and integrity checkings: check if database has PostGIS extensions, if the table exists, it is registered in RASTER_COLUMNS table, if has a GIST index, etc. Suggestions accepted and appreciated. |
| | 66 | 1. Fetch raster properties from RASTER_COLUMNS table. If the table is not registered in RASTER_COLUMNS, as we are working only in regularly_tiled mode, we should finish. |
| | 67 | 1. Populate its georeference information, to allow GetProjectionRef and GetGeoTransform methods provide correct information. |
| 102 | | ''The full set of possible GDALDataType values are declared in gdal.h, and include GDT_Byte, GDT_UInt16, GDT_Int16, and GDT_Float32''. All of them must be managed. |
| | 74 | 1. Get pixel size (now only 8bits). |
| | 75 | 1. Transform pixel,line coordinates into coordinates of the raster reference systems, by using the proper methods of Dataset, and get the coordinates of the block center. |
| | 76 | 1. Query for blocks that contains this point (the block center). As we only consider regularly_blocking rasters, the result will only 1 block. IMPORTANT: NOW, I'M USING A TESTING QUERY THAT ALWAYS FETCH THE SAME RASTER BLOCK. THE SPATIAL QUERY WILL BE ADDED ASAP. |
| | 77 | 1. Fetch the block in HEXWKB format and parse it to get the data. The HEXWKB format includes raster information header on each block, so, this information can be used for integrity checkings. |
| | 78 | 1. Copy the block data into the buffer. |
| 104 | | ''The block size is used to establish a natural or efficient block size to access the data with. For tiled datasets this will be the size of a tile, while for most other datasets it will be one scanline''. So, in our basic WKT Raster types reading, we should interpret this like ''the size of a tile''. |
| 105 | | |
| 106 | | And important issue here is ''how the tiles will finally be handled when fetching the raster data ''. Is the part in which I have more questions: |
| 107 | | * The data fetched with the Raster Band, is in HEXWKB format? (I think so) I should code a method to transform from this format to another ones... |
| 108 | | * Do we have to encode the data fetched in PNG/JPEG/TIFF format? |
| 109 | | * May we assume that the tiles stored at the same table are homogeneus? What does exactly 'homogeneus' mean here? Only same pixel type? |
| 110 | | * How do we manage the data fetched? Simply copy the raster data sequentially into the destination? Anf what happens if these raster data come from tiles with different pixeltype or band configuration? |
| 111 | | |
| 112 | | I think we should ''cut'', and take decissions about these questions. My proposal: |
| 113 | | * Data fetched in HEXWKB format. Implement a HEXWKB to binary decoding taking care to convert to local machine byte order if necessary. See OGRPGLayer::HEXToGeometry() method in ogrpglayer.cpp as a guide. |
| 114 | | * Apart from the possible implementation of out-db rasters, don't care about encoding to output format |
| 115 | | * ''In a regular_blocking table you can assume that all types are of the same pixel type, the same tile size (in pixels and lines), and the same geographic resolution.'' |
| 116 | | * ''When you establish the GDALRasterBand corresponding to a band in the table you will declare the pixel data type based on the contents of the RASTER_COLUMNS table. The IReadBlock() method will be invoked with a buffer that is the block/tile size and already of the declared pixel data type. The tile you get from the database should also be in this type (assuming regular_blocking). There should not be any translation of pixel type required though you will need to be byte order aware for non-byte image data. You will basically decode the HEXWKB image into the buffer passed into IReadBlock.'' |
| 117 | | |
| 118 | | Any suggestions? |
| 119 | | |
