| Version 4 (modified by , 9 years ago) ( diff ) |
|---|
RFC 59.1 : GDAL/OGR utilities as a library
Authors: Faza Mahamood, Even Rouault
Contact: fazamhd at gmail dot com, even.rouault at spatialys.com
Status: Adopted, implemented
Implementation version: 2.1
Summary
This RFC defines how to expose GDAL/OGR C/C++ utilities as C callable functions. The utility code is modified to call the new function. This RFC gives a general frame and principles, demonstrated with a few utilities, but aimed at being extended with other utilities.
Rationale
There is a need for calling GDAL utilities from code without involving system calls, to be able to work on in-memory datasets and use progress/cancellation callback functions.
Changes
A public header file gdal_utils.h is created which contains the public declarations of GDAL utilities. The current header(still in progress) can be found here.
Each utility has a function (XXXXOptionsNew) to create an option structure from arguments specified as an array of strings. This function also accepts as argument an extra semi-private structure used to cooperate with the code of the command line utility itself.
For GDALInfo():
/*! Options for GDALInfo(). Opaque type */ typedef struct GDALInfoOptions GDALInfoOptions; typedef struct GDALInfoOptionsForBinary GDALInfoOptionsForBinary; /** * Allocates a GDALInfoOptions struct. * * @param papszArgv NULL terminated list of options (potentially including filename and open options too) * The accepted options are the one of the gdalinfo utility. * @param psOptionsForBinary (output) may be NULL (and should generally be NULL), * otherwise (gdalinfo_bin.cpp use case) must be allocated with * GDALInfoOptionsForBinaryNew() prior to this function. Will be * filled with potentially present filename, open options, subdataset number... * @return pointer to the allocated GDALInfoOptions struct. * * @since GDAL 2.1 */ GDALInfoOptions CPL_DLL *GDALInfoOptionsNew(char** papszArgv, GDALInfoOptionsForBinary* psOptionsForBinary); void CPL_DLL GDALInfoOptionsFree( GDALInfoOptions *psOptions ); /** * Lists various information about a GDAL supported raster dataset. * * GDALInfoOptions* must be allocated and freed with GDALInfoOptionsNew() * and GDALInfoOptionsFree() respectively. * * @param hDataset the dataset handle. * @param psOptions the options structure returned by GDALInfoOptionsNew() or NULL. * @return string corresponding to the information about the raster dataset. * It must be freed using CPLFree(). * * @since GDAL 2.1 */ char CPL_DLL *GDALInfo( GDALDatasetH hDataset, const GDALInfoOptions *psOptions );
Similarly for GDALTranslate():
/*! Options for GDALTranslate(). Opaque type */
typedef struct GDALTranslateOptions GDALTranslateOptions;
typedef struct GDALTranslateOptionsForBinary GDALTranslateOptionsForBinary;
GDALTranslateOptions CPL_DLL *GDALTranslateOptionsNew(char** papszArgv,
GDALTranslateOptionsForBinary* psOptionsForBinary);
void CPL_DLL GDALTranslateOptionsFree( GDALTranslateOptions *psOptions );
void CPL_DLL GDALTranslateOptionsSetProgress( GDALTranslateOptions *psOptions,
GDALProgressFunc pfnProgress,
void *pProgressData );
GDALDatasetH CPL_DLL GDALTranslate(const char *pszDestFilename,
GDALDatasetH hSrcDataset,
const GDALTranslateOptions *psOptions,
int *pbUsageError);
Similarly for GDALWarp():
/*! Options for GDALWarp(). Opaque type */
typedef struct GDALWarpAppOptions GDALWarpAppOptions;
typedef struct GDALWarpAppOptionsForBinary GDALWarpAppOptionsForBinary;
GDALWarpAppOptions CPL_DLL *GDALWarpAppOptionsNew(char** papszArgv,
GDALWarpAppOptionsForBinary* psOptionsForBinary);
void CPL_DLL GDALWarpAppOptionsFree( GDALWarpAppOptions *psOptions );
void CPL_DLL GDALWarpAppOptionsSetProgress( GDALWarpAppOptions *psOptions,
GDALProgressFunc pfnProgress,
void *pProgressData );
void CPL_DLL GDALWarpAppOptionsSetWarpOption( GDALWarpAppOptions *psOptions,
const char* pszKey,
const char* pszValue );
GDALDatasetH CPL_DLL GDALWarp( const char *pszDest, GDALDatasetH hDstDS, int nSrcCount,
GDALDatasetH *pahSrcDS,
const GDALWarpAppOptions *psOptions, int *pbUsageError );
Similarly for GDALVectorTranslate() (equivalent of ogr2ogr):
/*! Options for GDALVectorTranslate(). Opaque type */
typedef struct GDALVectorTranslateOptions GDALVectorTranslateOptions;
typedef struct GDALVectorTranslateOptionsForBinary GDALVectorTranslateOptionsForBinary;
GDALVectorTranslateOptions CPL_DLL *GDALVectorTranslateOptionsNew(char** papszArgv,
GDALVectorTranslateOptionsForBinary* psOptionsForBinary);
void CPL_DLL GDALVectorTranslateOptionsFree( GDALVectorTranslateOptions *psOptions );
void CPL_DLL GDALVectorTranslateOptionsSetProgress( GDALVectorTranslateOptions *psOptions,
GDALProgressFunc pfnProgress,
void *pProgressData );
GDALDatasetH CPL_DLL GDALVectorTranslate( const char *pszDest, GDALDatasetH hDstDS, int nSrcCount,
GDALDatasetH *pahSrcDS,
const GDALVectorTranslateOptions *psOptions, int *pbUsageError );
SWIG bindings (Python / Java / C# / Perl) changes
For all bindings, the above functions are mapped to SWIG with :
struct GDALInfoOptions {
%extend {
GDALInfoOptions(char** options) {
return GDALInfoOptionsNew(options, NULL);
}
~GDALInfoOptions() {
GDALInfoOptionsFree( self );
}
}
};
%rename (InfoInternal) GDALInfo;
char *GDALInfo( GDALDatasetShadow *hDataset, GDALInfoOptions *infoOptions );
struct GDALTranslateOptions {
%extend {
GDALTranslateOptions(char** options) {
return GDALTranslateOptionsNew(options, NULL);
}
~GDALTranslateOptions() {
GDALTranslateOptionsFree( self );
}
}
};
%rename (TranslateInternal) wrapper_GDALTranslate;
%newobject wrapper_GDALTranslate;
%inline %{
GDALDatasetShadow* wrapper_GDALTranslate( const char* dest,
GDALDatasetShadow* dataset,
GDALTranslateOptions* translateOptions,
GDALProgressFunc callback=NULL,
void* callback_data=NULL);
struct GDALWarpAppOptions {
%extend {
GDALWarpAppOptions(char** options) {
return GDALWarpAppOptionsNew(options, NULL);
}
~GDALWarpAppOptions() {
GDALWarpAppOptionsFree( self );
}
}
};
/* Note: we must use 2 distinct names since there's a bug/feature in swig */
/* that doesn't play nicely with the (int object_list_count, GDALDatasetShadow** poObjects) input typemap */
%inline %{
int wrapper_GDALWarpDestDS( GDALDatasetShadow* dstDS,
int object_list_count, GDALDatasetShadow** poObjects,
GDALWarpAppOptions* warpAppOptions,
GDALProgressFunc callback=NULL,
void* callback_data=NULL),
%}
%newobject wrapper_GDALWarpDestName;
%inline %{
GDALDatasetShadow* wrapper_GDALWarpDestName( const char* dest,
int object_list_count, GDALDatasetShadow** poObjects,
GDALWarpAppOptions* warpAppOptions,
GDALProgressFunc callback=NULL,
void* callback_data=NULL),
%}
struct GDALVectorTranslateOptions {
%extend {
GDALVectorTranslateOptions(char** options) {
return GDALVectorTranslateOptionsNew(options, NULL);
}
~GDALVectorTranslateOptions() {
GDALVectorTranslateOptionsFree( self );
}
}
};
/* Note: we must use 2 distinct names since there's a bug/feature in swig */
/* that doesn't play nicely with the (int object_list_count, GDALDatasetShadow** poObjects) input typemap */
%inline %{
int wrapper_GDALVectorTranslateDestDS( GDALDatasetShadow* dstDS,
GDALDatasetShadow* srcDS,
GDALVectorTranslateOptions* options,
GDALProgressFunc callback=NULL,
void* callback_data=NULL);
%}
%newobject wrapper_GDALVectorTranslateDestName;
%inline %{
GDALDatasetShadow* wrapper_GDALVectorTranslateDestName( const char* dest,
GDALDatasetShadow* srcDS,
GDALVectorTranslateOptions* options,
GDALProgressFunc callback=NULL,
void* callback_data=NULL);
%}
For Python bindings, convenience wrappers are created to allow specifying options in a more user friendly way.
def InfoOptions(options = [], format = 'text', deserialize = True,
computeMinMax = False, reportHistograms = False, reportProj4 = False,
stats = False, approxStats = False, computeChecksum = False,
showGCPs = True, showMetadata = True, showRAT = True, showColorTable = True,
listMDD = False, showFileList = True, allMetadata = False,
extraMDDomains = None):
""" Create a InfoOptions() object that can be passed to gdal.Info()
options can be be an array of strings, a string or let empty and filled from other keywords."""
def Info(ds, **kwargs):
""" Return information on a dataset.
Arguments are :
ds --- a Dataset object or a filename
Keyword arguments are :
options --- return of gdal.InfoOptions(), string or array of strings
other keywords arguments of gdal.InfoOptions()
If options is provided as a gdal.InfoOptions() object, other keywords are ignored. """
def TranslateOptions(options = [], format = 'GTiff',
outputType = GDT_Unknown, bandList = None, maskBand = None,
width = 0, height = 0, widthPct = 0.0, heightPct = 0.0,
xRes = 0.0, yRes = 0.0,
creationOptions = None, srcWin = None, projWin = None, projWinSRS = None, strict = False,
unscale = False, scaleParams = None, exponents = None,
outputBounds = None, metadataOptions = None,
outputSRS = None, GCPs = None,
noData = None, rgbExpand = None,
stats = False, rat = True, resampleAlg = None,
callback = None, callback_data = None):
""" Create a TranslateOptions() object that can be passed to gdal.Translate()
Keyword arguments are :
options --- can be be an array of strings, a string or let empty and filled from other keywords.
format --- output format ("GTiff", etc...)
outputType --- output type (gdal.GDT_Byte, etc...)
bandList --- array of band numbers (index start at 1)
maskBand --- mask band to generate or not ("none", "auto", "mask", 1, ...)
width --- width of the output raster in pixel
height --- height of the output raster in pixel
widthPct --- width of the output raster in percentage (100 = original width)
heightPct --- height of the output raster in percentage (100 = original height)
xRes --- output horizontal resolution
yRes --- output vertical resolution
creationOptions --- list of creation options
srcWin --- subwindow in pixels to extract: [left_x, top_y, width, height]
projWin --- subwindow in projected coordinates to extract: [ulx, uly, lrx, lry]
projWinSRS --- SRS in which projWin is expressed
strict --- strict mode
unscale --- unscale values with scale and offset metadata
scaleParams --- list of scale parameters, each of the form [src_min,src_max] or [src_min,src_max,dst_min,dst_max]
exponents --- list of exponentation parameters
outputBounds --- assigned output bounds: [ulx, uly, lrx, lry]
metadataOptions --- list of metadata options
outputSRS --- assigned output SRS
GCPs --- list of GCPs
noData --- nodata value (or "none" to unset it)
rgbExpand --- Color palette expansion mode: "gray", "rgb", "rgba"
stats --- whether to calcule statistics
rat --- whether to write source RAT
resampleAlg --- resampling mode
callback --- callback method
callback_data --- user data for callback
"""
def Translate(destName, srcDS, **kwargs):
""" Convert a dataset.
Arguments are :
destName --- Output dataset name
srcDS --- a Dataset object or a filename
Keyword arguments are :
options --- return of gdal.InfoOptions(), string or array of strings
other keywords arguments of gdal.TranslateOptions()
If options is provided as a gdal.TranslateOptions() object, other keywords are ignored. """
def WarpOptions(options = [], format = 'GTiff',
outputBounds = None,
outputBoundsSRS = None,
xRes = None, yRes = None, targetAlignedPixels = False,
width = 0, height = 0,
srcSRS = None, dstSRS = None,
srcAlpha = False, dstAlpha = False,
warpOptions = None, errorThreshold = None,
warpMemoryLimit = None, creationOptions = None, outputType = GDT_Unknown,
workingType = GDT_Unknown, resampleAlg = None,
srcNodata = None, dstNodata = None, multithread = False,
tps = False, rpc = False, geoloc = False, polynomialOrder = None,
transformerOptions = None, cutlineDSName = None,
cutlineLayer = None, cutlineWhere = None, cutlineSQL = None, cutlineBlend = None, cropToCutline = False,
copyMetadata = True, metadataConflictValue = None,
setColorInterpretation = False,
callback = None, callback_data = None):
""" Create a WarpOptions() object that can be passed to gdal.Warp()
Keyword arguments are :
options --- can be be an array of strings, a string or let empty and filled from other keywords.
format --- output format ("GTiff", etc...)
outputBounds --- output bounds as (minX, minY, maxX, maxY) in target SRS
outputBoundsSRS --- SRS in which output bounds are expressed, in the case they are not expressed in dstSRS
xRes, yRes --- output resolution in target SRS
targetAlignedPixels --- whether to force output bounds to be multiple of output resolution
width --- width of the output raster in pixel
height --- height of the output raster in pixel
srcSRS --- source SRS
dstSRS --- output SRS
srcAlpha --- whether to force the last band of the input dataset to be considered as an alpha band
dstAlpha --- whether to force the creation of an output alpha band
outputType --- output type (gdal.GDT_Byte, etc...)
workingType --- working type (gdal.GDT_Byte, etc...)
warpOptions --- list of warping options
errorThreshold --- error threshold for approximation transformer (in pixels)
warpMemoryLimit --- size of working buffer in bytes
resampleAlg --- resampling mode
creationOptions --- list of creation options
srcNodata --- source nodata value(s)
dstNodata --- output nodata value(s)
multithread --- whether to multithread computation and I/O operations
tps --- whether to use Thin Plate Spline GCP transformer
rpc --- whether to use RPC transformer
geoloc --- whether to use GeoLocation array transformer
polynomialOrder --- order of polynomial GCP interpolation
transformerOptions --- list of transformer options
cutlineDSName --- cutline dataset name
cutlineLayer --- cutline layer name
cutlineWhere --- cutline WHERE clause
cutlineSQL --- cutline SQL statement
cutlineBlend --- cutline blend distance in pixels
cropToCutline --- whether to use cutline extent for output bounds
copyMetadata --- whether to copy source metadata
metadataConflictValue --- metadata data conflict value
setColorInterpretation --- whether to force color interpretation of input bands to output bands
callback --- callback method
callback_data --- user data for callback
"""
def Warp(destNameOrDestDS, srcDSOrSrcDSTab, **kwargs):
""" Warp one or several datasets.
Arguments are :
destNameOrDestDS --- Output dataset name or object
srcDSOrSrcDSTab --- an array of Dataset objects or filenames, or a Dataset object or a filename
Keyword arguments are :
options --- return of gdal.InfoOptions(), string or array of strings
other keywords arguments of gdal.WarpOptions()
If options is provided as a gdal.WarpOptions() object, other keywords are ignored. """
def VectorTranslateOptions(options = [], format = 'ESRI Shapefile',
accessMode = None,
srcSRS = None, dstSRS = None, reproject = True,
SQLStatement = None, SQLDialect = None, where = None, selectFields = None, spatFilter = None,
datasetCreationOptions = None,
layerCreationOptions = None,
layers = None,
layerName = None,
geometryType = None,
segmentizeMaxDist= None,
callback = None, callback_data = None):
""" Create a VectorTranslateOptions() object that can be passed to gdal.VectorTranslate()
Keyword arguments are :
options --- can be be an array of strings, a string or let empty and filled from other keywords.
format --- output format ("ESRI Shapefile", etc...)
accessMode --- None for creation, 'update', 'append', 'overwrite'
srcSRS --- source SRS
dstSRS --- output SRS (with reprojection if reproject = True)
reproject --- whether to do reprojection
SQLStatement --- SQL statement to apply to the source dataset
SQLDialect --- SQL dialect ('OGRSQL', 'SQLITE', ...)
where --- WHERE clause to apply to source layer(s)
selectFields --- list of fields to select
spatFilter --- spatial filter as (minX, minY, maxX, maxY) bounding box
datasetCreationOptions --- list of dataset creation options
layerCreationOptions --- list of layer creation options
layers --- list of layers to convert
layerName --- output layer name
geometryType --- output layer geometry type ('POINT', ....)
segmentizeMaxDist --- maximum distance between consecutive nodes of a line geometry
callback --- callback method
callback_data --- user data for callback
"""
def VectorTranslate(destNameOrDestDS, srcDS, **kwargs):
""" Convert one vector dataset
Arguments are :
destNameOrDestDS --- Output dataset name or object
srcDS --- a Dataset object or a filename
Keyword arguments are :
options --- return of gdal.InfoOptions(), string or array of strings
other keywords arguments of gdal.VectorTranslateOptions()
If options is provided as a gdal.VectorTranslateOptions() object, other keywords are ignored. """
gdal.Info() can be used either with setting the attributes of gdal.InfoOptions() or inlined arguments of gdal.Info(). Arguments can be specified as array of strings, command line syntax or dedeicated keywords. So various combinations are possible :
options = gdal.InfoOptions(format = 'json', computeChecksum = True)
gdal.Info(ds, options)
options = gdal.InfoOptions(options = ['-json', '-checksum'])
gdal.Info(ds, options)
options = gdal.InfoOptions(options = '-json -checksum')
gdal.Info(ds, options)
gdal.Info(ds, format = 'json', computeChecksum = True)
gdal.Info(ds, options = ['-json', '-checksum'])
gdal.Info(ds, options = '-json -checksum')
Same for gdal.Translate()
Utilities
Utilities are modified to call the respective function.
Documentation
All new methods/functions are documented.
Test Suite
gdal.Info method is tested in test_gdalinfo_lib.py. gdal.Translate method is tested in test_gdal_translate_lib.py. gdal.Warp method is tested in test_gdalwarp_lib.py. gdal.VectorTranslate method is tested in test_ogr2ogr_lib.py.
Compatibility Issues
None expected. Command line utilities will keep the same interface. It will be checked by ensuring their tests in autotest/utilities still pass.
Related ticket
Implementation
Implementation will be done by Faza Mahamood and Even Rouault
The proposed implementation lies in the "rfc59.1" branch of the https://github.com/rouault/gdal2/tree/rfc59.1.
The list of changes : https://github.com/rouault/gdal2/compare/rfc59.1
Voting history
+1 from DanielM and EvenR
Note:
See TracWiki
for help on using the wiki.
