Version 4 (modified by 9 years ago) ( diff ) | ,
---|
RFC 59.1 : GDAL/OGR utilities as a library
Authors: Faza Mahamood, Even Rouault
Contact: fazamhd at gmail dot com, even.rouault at spatialys.com
Status: Adopted, implemented
Implementation version: 2.1
Summary
This RFC defines how to expose GDAL/OGR C/C++ utilities as C callable functions. The utility code is modified to call the new function. This RFC gives a general frame and principles, demonstrated with a few utilities, but aimed at being extended with other utilities.
Rationale
There is a need for calling GDAL utilities from code without involving system calls, to be able to work on in-memory datasets and use progress/cancellation callback functions.
Changes
A public header file gdal_utils.h is created which contains the public declarations of GDAL utilities. The current header(still in progress) can be found here.
Each utility has a function (XXXXOptionsNew) to create an option structure from arguments specified as an array of strings. This function also accepts as argument an extra semi-private structure used to cooperate with the code of the command line utility itself.
For GDALInfo():
/*! Options for GDALInfo(). Opaque type */ typedef struct GDALInfoOptions GDALInfoOptions; typedef struct GDALInfoOptionsForBinary GDALInfoOptionsForBinary; /** * Allocates a GDALInfoOptions struct. * * @param papszArgv NULL terminated list of options (potentially including filename and open options too) * The accepted options are the one of the gdalinfo utility. * @param psOptionsForBinary (output) may be NULL (and should generally be NULL), * otherwise (gdalinfo_bin.cpp use case) must be allocated with * GDALInfoOptionsForBinaryNew() prior to this function. Will be * filled with potentially present filename, open options, subdataset number... * @return pointer to the allocated GDALInfoOptions struct. * * @since GDAL 2.1 */ GDALInfoOptions CPL_DLL *GDALInfoOptionsNew(char** papszArgv, GDALInfoOptionsForBinary* psOptionsForBinary); void CPL_DLL GDALInfoOptionsFree( GDALInfoOptions *psOptions ); /** * Lists various information about a GDAL supported raster dataset. * * GDALInfoOptions* must be allocated and freed with GDALInfoOptionsNew() * and GDALInfoOptionsFree() respectively. * * @param hDataset the dataset handle. * @param psOptions the options structure returned by GDALInfoOptionsNew() or NULL. * @return string corresponding to the information about the raster dataset. * It must be freed using CPLFree(). * * @since GDAL 2.1 */ char CPL_DLL *GDALInfo( GDALDatasetH hDataset, const GDALInfoOptions *psOptions );
Similarly for GDALTranslate():
/*! Options for GDALTranslate(). Opaque type */ typedef struct GDALTranslateOptions GDALTranslateOptions; typedef struct GDALTranslateOptionsForBinary GDALTranslateOptionsForBinary; GDALTranslateOptions CPL_DLL *GDALTranslateOptionsNew(char** papszArgv, GDALTranslateOptionsForBinary* psOptionsForBinary); void CPL_DLL GDALTranslateOptionsFree( GDALTranslateOptions *psOptions ); void CPL_DLL GDALTranslateOptionsSetProgress( GDALTranslateOptions *psOptions, GDALProgressFunc pfnProgress, void *pProgressData ); GDALDatasetH CPL_DLL GDALTranslate(const char *pszDestFilename, GDALDatasetH hSrcDataset, const GDALTranslateOptions *psOptions, int *pbUsageError);
Similarly for GDALWarp():
/*! Options for GDALWarp(). Opaque type */ typedef struct GDALWarpAppOptions GDALWarpAppOptions; typedef struct GDALWarpAppOptionsForBinary GDALWarpAppOptionsForBinary; GDALWarpAppOptions CPL_DLL *GDALWarpAppOptionsNew(char** papszArgv, GDALWarpAppOptionsForBinary* psOptionsForBinary); void CPL_DLL GDALWarpAppOptionsFree( GDALWarpAppOptions *psOptions ); void CPL_DLL GDALWarpAppOptionsSetProgress( GDALWarpAppOptions *psOptions, GDALProgressFunc pfnProgress, void *pProgressData ); void CPL_DLL GDALWarpAppOptionsSetWarpOption( GDALWarpAppOptions *psOptions, const char* pszKey, const char* pszValue ); GDALDatasetH CPL_DLL GDALWarp( const char *pszDest, GDALDatasetH hDstDS, int nSrcCount, GDALDatasetH *pahSrcDS, const GDALWarpAppOptions *psOptions, int *pbUsageError );
Similarly for GDALVectorTranslate() (equivalent of ogr2ogr):
/*! Options for GDALVectorTranslate(). Opaque type */ typedef struct GDALVectorTranslateOptions GDALVectorTranslateOptions; typedef struct GDALVectorTranslateOptionsForBinary GDALVectorTranslateOptionsForBinary; GDALVectorTranslateOptions CPL_DLL *GDALVectorTranslateOptionsNew(char** papszArgv, GDALVectorTranslateOptionsForBinary* psOptionsForBinary); void CPL_DLL GDALVectorTranslateOptionsFree( GDALVectorTranslateOptions *psOptions ); void CPL_DLL GDALVectorTranslateOptionsSetProgress( GDALVectorTranslateOptions *psOptions, GDALProgressFunc pfnProgress, void *pProgressData ); GDALDatasetH CPL_DLL GDALVectorTranslate( const char *pszDest, GDALDatasetH hDstDS, int nSrcCount, GDALDatasetH *pahSrcDS, const GDALVectorTranslateOptions *psOptions, int *pbUsageError );
SWIG bindings (Python / Java / C# / Perl) changes
For all bindings, the above functions are mapped to SWIG with :
struct GDALInfoOptions { %extend { GDALInfoOptions(char** options) { return GDALInfoOptionsNew(options, NULL); } ~GDALInfoOptions() { GDALInfoOptionsFree( self ); } } }; %rename (InfoInternal) GDALInfo; char *GDALInfo( GDALDatasetShadow *hDataset, GDALInfoOptions *infoOptions );
struct GDALTranslateOptions { %extend { GDALTranslateOptions(char** options) { return GDALTranslateOptionsNew(options, NULL); } ~GDALTranslateOptions() { GDALTranslateOptionsFree( self ); } } }; %rename (TranslateInternal) wrapper_GDALTranslate; %newobject wrapper_GDALTranslate; %inline %{ GDALDatasetShadow* wrapper_GDALTranslate( const char* dest, GDALDatasetShadow* dataset, GDALTranslateOptions* translateOptions, GDALProgressFunc callback=NULL, void* callback_data=NULL);
struct GDALWarpAppOptions { %extend { GDALWarpAppOptions(char** options) { return GDALWarpAppOptionsNew(options, NULL); } ~GDALWarpAppOptions() { GDALWarpAppOptionsFree( self ); } } }; /* Note: we must use 2 distinct names since there's a bug/feature in swig */ /* that doesn't play nicely with the (int object_list_count, GDALDatasetShadow** poObjects) input typemap */ %inline %{ int wrapper_GDALWarpDestDS( GDALDatasetShadow* dstDS, int object_list_count, GDALDatasetShadow** poObjects, GDALWarpAppOptions* warpAppOptions, GDALProgressFunc callback=NULL, void* callback_data=NULL), %} %newobject wrapper_GDALWarpDestName; %inline %{ GDALDatasetShadow* wrapper_GDALWarpDestName( const char* dest, int object_list_count, GDALDatasetShadow** poObjects, GDALWarpAppOptions* warpAppOptions, GDALProgressFunc callback=NULL, void* callback_data=NULL), %}
struct GDALVectorTranslateOptions { %extend { GDALVectorTranslateOptions(char** options) { return GDALVectorTranslateOptionsNew(options, NULL); } ~GDALVectorTranslateOptions() { GDALVectorTranslateOptionsFree( self ); } } }; /* Note: we must use 2 distinct names since there's a bug/feature in swig */ /* that doesn't play nicely with the (int object_list_count, GDALDatasetShadow** poObjects) input typemap */ %inline %{ int wrapper_GDALVectorTranslateDestDS( GDALDatasetShadow* dstDS, GDALDatasetShadow* srcDS, GDALVectorTranslateOptions* options, GDALProgressFunc callback=NULL, void* callback_data=NULL); %} %newobject wrapper_GDALVectorTranslateDestName; %inline %{ GDALDatasetShadow* wrapper_GDALVectorTranslateDestName( const char* dest, GDALDatasetShadow* srcDS, GDALVectorTranslateOptions* options, GDALProgressFunc callback=NULL, void* callback_data=NULL); %}
For Python bindings, convenience wrappers are created to allow specifying options in a more user friendly way.
def InfoOptions(options = [], format = 'text', deserialize = True, computeMinMax = False, reportHistograms = False, reportProj4 = False, stats = False, approxStats = False, computeChecksum = False, showGCPs = True, showMetadata = True, showRAT = True, showColorTable = True, listMDD = False, showFileList = True, allMetadata = False, extraMDDomains = None): """ Create a InfoOptions() object that can be passed to gdal.Info() options can be be an array of strings, a string or let empty and filled from other keywords.""" def Info(ds, **kwargs): """ Return information on a dataset. Arguments are : ds --- a Dataset object or a filename Keyword arguments are : options --- return of gdal.InfoOptions(), string or array of strings other keywords arguments of gdal.InfoOptions() If options is provided as a gdal.InfoOptions() object, other keywords are ignored. """
def TranslateOptions(options = [], format = 'GTiff', outputType = GDT_Unknown, bandList = None, maskBand = None, width = 0, height = 0, widthPct = 0.0, heightPct = 0.0, xRes = 0.0, yRes = 0.0, creationOptions = None, srcWin = None, projWin = None, projWinSRS = None, strict = False, unscale = False, scaleParams = None, exponents = None, outputBounds = None, metadataOptions = None, outputSRS = None, GCPs = None, noData = None, rgbExpand = None, stats = False, rat = True, resampleAlg = None, callback = None, callback_data = None): """ Create a TranslateOptions() object that can be passed to gdal.Translate() Keyword arguments are : options --- can be be an array of strings, a string or let empty and filled from other keywords. format --- output format ("GTiff", etc...) outputType --- output type (gdal.GDT_Byte, etc...) bandList --- array of band numbers (index start at 1) maskBand --- mask band to generate or not ("none", "auto", "mask", 1, ...) width --- width of the output raster in pixel height --- height of the output raster in pixel widthPct --- width of the output raster in percentage (100 = original width) heightPct --- height of the output raster in percentage (100 = original height) xRes --- output horizontal resolution yRes --- output vertical resolution creationOptions --- list of creation options srcWin --- subwindow in pixels to extract: [left_x, top_y, width, height] projWin --- subwindow in projected coordinates to extract: [ulx, uly, lrx, lry] projWinSRS --- SRS in which projWin is expressed strict --- strict mode unscale --- unscale values with scale and offset metadata scaleParams --- list of scale parameters, each of the form [src_min,src_max] or [src_min,src_max,dst_min,dst_max] exponents --- list of exponentation parameters outputBounds --- assigned output bounds: [ulx, uly, lrx, lry] metadataOptions --- list of metadata options outputSRS --- assigned output SRS GCPs --- list of GCPs noData --- nodata value (or "none" to unset it) rgbExpand --- Color palette expansion mode: "gray", "rgb", "rgba" stats --- whether to calcule statistics rat --- whether to write source RAT resampleAlg --- resampling mode callback --- callback method callback_data --- user data for callback """ def Translate(destName, srcDS, **kwargs): """ Convert a dataset. Arguments are : destName --- Output dataset name srcDS --- a Dataset object or a filename Keyword arguments are : options --- return of gdal.InfoOptions(), string or array of strings other keywords arguments of gdal.TranslateOptions() If options is provided as a gdal.TranslateOptions() object, other keywords are ignored. """
def WarpOptions(options = [], format = 'GTiff', outputBounds = None, outputBoundsSRS = None, xRes = None, yRes = None, targetAlignedPixels = False, width = 0, height = 0, srcSRS = None, dstSRS = None, srcAlpha = False, dstAlpha = False, warpOptions = None, errorThreshold = None, warpMemoryLimit = None, creationOptions = None, outputType = GDT_Unknown, workingType = GDT_Unknown, resampleAlg = None, srcNodata = None, dstNodata = None, multithread = False, tps = False, rpc = False, geoloc = False, polynomialOrder = None, transformerOptions = None, cutlineDSName = None, cutlineLayer = None, cutlineWhere = None, cutlineSQL = None, cutlineBlend = None, cropToCutline = False, copyMetadata = True, metadataConflictValue = None, setColorInterpretation = False, callback = None, callback_data = None): """ Create a WarpOptions() object that can be passed to gdal.Warp() Keyword arguments are : options --- can be be an array of strings, a string or let empty and filled from other keywords. format --- output format ("GTiff", etc...) outputBounds --- output bounds as (minX, minY, maxX, maxY) in target SRS outputBoundsSRS --- SRS in which output bounds are expressed, in the case they are not expressed in dstSRS xRes, yRes --- output resolution in target SRS targetAlignedPixels --- whether to force output bounds to be multiple of output resolution width --- width of the output raster in pixel height --- height of the output raster in pixel srcSRS --- source SRS dstSRS --- output SRS srcAlpha --- whether to force the last band of the input dataset to be considered as an alpha band dstAlpha --- whether to force the creation of an output alpha band outputType --- output type (gdal.GDT_Byte, etc...) workingType --- working type (gdal.GDT_Byte, etc...) warpOptions --- list of warping options errorThreshold --- error threshold for approximation transformer (in pixels) warpMemoryLimit --- size of working buffer in bytes resampleAlg --- resampling mode creationOptions --- list of creation options srcNodata --- source nodata value(s) dstNodata --- output nodata value(s) multithread --- whether to multithread computation and I/O operations tps --- whether to use Thin Plate Spline GCP transformer rpc --- whether to use RPC transformer geoloc --- whether to use GeoLocation array transformer polynomialOrder --- order of polynomial GCP interpolation transformerOptions --- list of transformer options cutlineDSName --- cutline dataset name cutlineLayer --- cutline layer name cutlineWhere --- cutline WHERE clause cutlineSQL --- cutline SQL statement cutlineBlend --- cutline blend distance in pixels cropToCutline --- whether to use cutline extent for output bounds copyMetadata --- whether to copy source metadata metadataConflictValue --- metadata data conflict value setColorInterpretation --- whether to force color interpretation of input bands to output bands callback --- callback method callback_data --- user data for callback """ def Warp(destNameOrDestDS, srcDSOrSrcDSTab, **kwargs): """ Warp one or several datasets. Arguments are : destNameOrDestDS --- Output dataset name or object srcDSOrSrcDSTab --- an array of Dataset objects or filenames, or a Dataset object or a filename Keyword arguments are : options --- return of gdal.InfoOptions(), string or array of strings other keywords arguments of gdal.WarpOptions() If options is provided as a gdal.WarpOptions() object, other keywords are ignored. """
def VectorTranslateOptions(options = [], format = 'ESRI Shapefile', accessMode = None, srcSRS = None, dstSRS = None, reproject = True, SQLStatement = None, SQLDialect = None, where = None, selectFields = None, spatFilter = None, datasetCreationOptions = None, layerCreationOptions = None, layers = None, layerName = None, geometryType = None, segmentizeMaxDist= None, callback = None, callback_data = None): """ Create a VectorTranslateOptions() object that can be passed to gdal.VectorTranslate() Keyword arguments are : options --- can be be an array of strings, a string or let empty and filled from other keywords. format --- output format ("ESRI Shapefile", etc...) accessMode --- None for creation, 'update', 'append', 'overwrite' srcSRS --- source SRS dstSRS --- output SRS (with reprojection if reproject = True) reproject --- whether to do reprojection SQLStatement --- SQL statement to apply to the source dataset SQLDialect --- SQL dialect ('OGRSQL', 'SQLITE', ...) where --- WHERE clause to apply to source layer(s) selectFields --- list of fields to select spatFilter --- spatial filter as (minX, minY, maxX, maxY) bounding box datasetCreationOptions --- list of dataset creation options layerCreationOptions --- list of layer creation options layers --- list of layers to convert layerName --- output layer name geometryType --- output layer geometry type ('POINT', ....) segmentizeMaxDist --- maximum distance between consecutive nodes of a line geometry callback --- callback method callback_data --- user data for callback """ def VectorTranslate(destNameOrDestDS, srcDS, **kwargs): """ Convert one vector dataset Arguments are : destNameOrDestDS --- Output dataset name or object srcDS --- a Dataset object or a filename Keyword arguments are : options --- return of gdal.InfoOptions(), string or array of strings other keywords arguments of gdal.VectorTranslateOptions() If options is provided as a gdal.VectorTranslateOptions() object, other keywords are ignored. """
gdal.Info() can be used either with setting the attributes of gdal.InfoOptions() or inlined arguments of gdal.Info(). Arguments can be specified as array of strings, command line syntax or dedeicated keywords. So various combinations are possible :
options = gdal.InfoOptions(format = 'json', computeChecksum = True) gdal.Info(ds, options)
options = gdal.InfoOptions(options = ['-json', '-checksum']) gdal.Info(ds, options)
options = gdal.InfoOptions(options = '-json -checksum') gdal.Info(ds, options)
gdal.Info(ds, format = 'json', computeChecksum = True)
gdal.Info(ds, options = ['-json', '-checksum'])
gdal.Info(ds, options = '-json -checksum')
Same for gdal.Translate()
Utilities
Utilities are modified to call the respective function.
Documentation
All new methods/functions are documented.
Test Suite
gdal.Info method is tested in test_gdalinfo_lib.py. gdal.Translate method is tested in test_gdal_translate_lib.py. gdal.Warp method is tested in test_gdalwarp_lib.py. gdal.VectorTranslate method is tested in test_ogr2ogr_lib.py.
Compatibility Issues
None expected. Command line utilities will keep the same interface. It will be checked by ensuring their tests in autotest/utilities still pass.
Related ticket
Implementation
Implementation will be done by Faza Mahamood and Even Rouault
The proposed implementation lies in the "rfc59.1" branch of the https://github.com/rouault/gdal2/tree/rfc59.1.
The list of changes : https://github.com/rouault/gdal2/compare/rfc59.1
Voting history
+1 from DanielM and EvenR