Opened 8 years ago

Closed 5 years ago

#3228 closed enhancement (wontfix)

PostGIS Raster driver: Block caching

Reported by: jorgearevalo Owned by: jorgearevalo
Priority: high Milestone: 1.9.1
Component: GDAL_Raster Version: svn-trunk
Severity: normal Keywords: postgis_raster, postgisraster, wktraster
Cc:

Description

Is necessary a caching method to avoid fetching data from database each time IReadBlock is called.

Change History (15)

comment:1 Changed 8 years ago by jorgearevalo

Status: newassigned

comment:2 Changed 7 years ago by jorgearevalo

Keywords: postgis raster added; wkt removed
Milestone: 1.7.01.8.0
Summary: WKT Raster driver: Block cachingPostGIS Raster driver: Block caching

comment:3 Changed 6 years ago by jorgearevalo

Milestone: 1.8.11.8.2

Implementing IRasterIO (at dataset level) we avoid too many server rounds by fetching all the required tiles in one query. The problem is not all tiles has to have the same dimensions. This complicates the use of GDAL cache. We can try to solve it by divinding all the fetched data in equally sized blocks, and manually storing them on the cache.

comment:4 Changed 6 years ago by jorgearevalo

Basic caching on the easiest case (regulary blocked raster, region size equal to buffer size) implemented in r23098. Working on known bugs and limitations.

comment:5 Changed 6 years ago by jorgearevalo

Bug related with non-square tiles. 2 use cases cause errors (image looks like a puzzle with the pieces in wrong places):

  • Image divided into non-square tiles
  • Non-square image divided into square tiles (or non-square tiles, it fails anyway)

comment:6 Changed 6 years ago by jorgearevalo

The bug wasn't related with non-square tiles. The problem was PostGISRasterDataset::IRasterIO puts all the blocks in the output buffer in the same order they were fetched from database, and this order isn't necessarily be the 'right' order, in terms of position inside the raster.

The temporary solution is to force a order by rid in the query that fetches the blocks from database. This is not the perfect solution, because the blocks are not stored in a particular order (the rid order normally matches the position of each block inside the image, but this is not a constraint). This temporary solution has been applied at r23180

comment:7 Changed 6 years ago by jorgearevalo

With r23218, the tiles are fetched following a upperlefty, upperleftx order. Now, it's necessary to ensure a good block size, that works in any situation (regular and irregular blocking).

comment:8 Changed 6 years ago by jorgearevalo

With r23219, we check that all the blocks have the same size, snap to the same grid and have the same srid. If one of these fail, we raise and error. Defined attributes to allow user to specify these constraints are true by default, and avoid the checkings.

comment:9 Changed 6 years ago by warmerdam

Milestone: 1.8.2

removing milestone.

comment:10 Changed 6 years ago by pracine

Keywords: postgis_raster added; postgis raster removed

comment:11 Changed 6 years ago by jorgearevalo

Milestone: 1.9.1

comment:12 Changed 5 years ago by Mateusz Łoskot

Keywords: postgisraster wktraster added

comment:13 Changed 5 years ago by jorgearevalo

Resolution: fixed
Status: assignedclosed

The problem is avoided using MEM/VRT drivers to take care of the data. Done in r24815

comment:14 Changed 5 years ago by jorgearevalo

Resolution: fixed
Status: closedreopened

comment:15 Changed 5 years ago by jorgearevalo

Resolution: wontfix
Status: reopenedclosed
Note: See TracTickets for help on using tickets.