Changes between Initial Version and Version 3 of Ticket #4979


Ignore:
Timestamp:
Feb 6, 2013, 11:24:15 AM (11 years ago)
Author:
Even Rouault
Comment:

Releated commits : r25601, r25602, r25603, r25605, r25606

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #4979

    • Property Summary Implement a GDAL RPC protocol to make GDAL (more) failsafe → Implement a GDAL API Proxy mechanism to make GDAL (more) failsafe

  • Ticket #4979 – Description

    initial v3  
     1
     2Introduction
     3
     4(GDAL >= 1.10.0)
     5
    16When dealing with some file formats, particularly the drivers relying on third-party (potentially closed-source) libraries, it is difficult to ensure that those third-party libraries will be robust to hostile/corrupted datasource.
    27
    3 The implemented solution is to have a fake driver that will expose a GDALClientDataset object that will forward all the API calls to another process ("server") where the real driver will be run. This way, if the server segfaults, the calling process will be unaffected and will report a clean error instead of segfaulting itself.
     8The implemented solution is to have a (private) API_PROXY driver that will expose a GDALClientDataset object, which will forward all the GDAL API calls to another process ("server"), where the real driver will be effectively run. This way, if the server aborts due to a fatal error, the calling process will be unaffected and will report a clean error instead of aborting itself.
    49
    5 This is controlled by setting the GDAL_RPC config option to YES. The option can also be set to a list of file extensions that must be the only ones to trigger this mechanism (e.g. GDAL_RPC=ecw,sid)
    610
    7 For now, the server launched is the gdalserver executable on Windows. On Unix, the default behaviour is to just fork() the current process (the gdalserver executable can also be execvp()'ed by forcing GDAL_RPC_SERVER=YES).
     11How to enable ?
    812
    9 To avoid the cost of forking()/CreateProcess(), a pool of connections is used. This is controlled with the GDAL_RPC_RECYCLE config option that is set to YES by default.
     13The API_PROXY mechanism can be enabled by setting the GDAL_API_PROXY config option to YES. The option can also be set to a list of file extensions that must be the only ones to trigger this mechanism (e.g. GDAL_API_PROXY=ecw,sid).
    1014
    11 Note: the client/server approach can also be used for thread unsafe drivers (not actually tested).
     15When enabled, datasets can be handled with GDALOpen(), GDALCreate() or GDALCreateCopy() with their nominal filename (or connexion string).
     16
     17Alternatively, the API_PROXY mechanism can be used selectively on a datasource by prefixing its name with API_PROXY:, for example GDALOpen("API_PROXY:foo.tif", GA_ReadOnly).
     18
     19
     20Advanced options
     21
     22For now, the server launched is the gdalserver executable on Windows. On Unix, the default behaviour is to just fork() the current process. It is also possible to launch the gdalserver executable by forcing GDAL_API_PROXY_SERVER=YES. The full filename of the gdalserver executable can also be specified in the GDAL_API_PROXY_SERVER.
     23
     24In case of many dataset opening or creation, to avoid the cost of repeated process forking, a pool of unused connections is established. Each time a dataset is closed, the associated connection is pushed in the pool (if there's an empty bucket). When a following dataset is to be opened, one of those connections will be reused. This behaviour is controlled with the GDAL_API_PROXY_CONN_POOL config option that is set to YES by default, and will keep a maximum of 4 unused connections. GDAL_API_PROXY_CONN_POOL can be set to a integer value to specify the maximum number of unused connections.
     25
     26
     27Limitations
     28
     29Datasets stored in the memory virtual file system (/vsimem) or handled by the MEM driver are excluded from the API Proxy mechanism.
     30
     31Additionnaly, for Create() or CreateCopy(), the VRT driver is also excluded from that mechanism.
     32