#ifndef DOXYGEN_SKIP /* $Id: vrt_tutorial.dox 41777 2018-03-13 16:39:33Z rouault $ */ #endif /* DOXYGEN_SKIP */ /*! \page gdal_vrttut GDAL Virtual Format Tutorial \section gdal_vrttut_toc Contents
  1. \ref gdal_vrttut_intro
  2. \ref gdal_vrttut_format
  3. \ref gdal_vrttut_overviews
  4. \ref gdal_vrttut_raw
  5. \ref gdal_vrttut_creation
  6. \ref gdal_vrttut_derived_c
  7. \ref gdal_vrttut_derived_python
  8. \ref gdal_vrttut_warped
  9. \ref gdal_vrttut_pansharpen
  10. \ref gdal_vrttut_mt
  11. \ref gdal_vrttut_perf
\section gdal_vrttut_intro Introduction The VRT driver is a format driver for GDAL that allows a virtual GDAL dataset to be composed from other GDAL datasets with repositioning, and algorithms potentially applied as well as various kinds of metadata altered or added. VRT descriptions of datasets can be saved in an XML format normally given the extension .vrt.

The VRT format can also describe \ref gdal_vrttut_warped "warping operations" and \ref gdal_vrttut_pansharpen "pansharpening operations".

An example of a simple .vrt file referring to a 512x512 dataset with one band loaded from utm.tif might look like this: \code 440720.0, 60.0, 0.0, 3751320.0, 0.0, -60.0 Gray utm.tif 1 \endcode Many aspects of the VRT file are a direct XML encoding of the GDAL Data Model which should be reviewed for understanding of the semantics of various elements.

VRT files can be produced by translating to VRT format. The resulting file can then be edited to modify mappings, add metadata or other purposes. VRT files can also be produced programmatically by various means.

This tutorial will cover the .vrt file format (suitable for users editing .vrt files), and how .vrt files may be created and manipulated programmatically for developers.

\section gdal_vrttut_format .vrt Format A XML schema of the GDAL VRT format is available.

Virtual files stored on disk are kept in an XML format with the following elements.

VRTDataset: This is the root element for the whole GDAL dataset. It must have the attributes rasterXSize and rasterYSize describing the width and height of the dataset in pixels. It may have a subClass attributes with values VRTWarpedDataset (\ref gdal_vrttut_warped) or VRTPansharpenedDataset (\ref gdal_vrttut_pansharpen). It may have SRS, GeoTransform, GCPList, Metadata, MaskBand and VRTRasterBand subelements. \code \endcode The allowed subelements for VRTDataset are :

\section gdal_vrttut_overviews Overviews GDAL can make efficient use of overviews available in the sources that compose the bands when dealing with RasterIO() requests that involve downsampling. But in the general case, the VRT bands themselves will not expose overviews. Except if (from top priority to lesser priority) :
  1. The Overview element is present in the VRTRasterBand element. See above.
  2. or external .vrt.ovr overviews are built
  3. (starting with GDAL 2.1) if the VRTRasterBand are made of a single SimpleSource or ComplexSource that has overviews. Those "virtual" overviews will be hidden by external .vrt.ovr overviews that might be built later.
\section gdal_vrttut_raw .vrt Descriptions for Raw Files So far we have described how to derive new virtual datasets from existing files supports by GDAL. However, it is also common to need to utilize raw binary raster files for which the regular layout of the data is known but for which no format specific driver exists. This can be accomplished by writing a .vrt file describing the raw file. For example, the following .vrt describes a raw raster file containing floating point complex pixels in a file called l2p3hhsso.img. The image data starts from the first byte (ImageOffset=0). The byte offset between pixels is 8 (PixelOffset=8), the size of a CFloat32. The byte offset from the start of one line to the start of the next is 9376 bytes (LineOffset=9376) which is the width (1172) times the size of a pixel (8). \code l2p3hhsso.img 0 8 9376 MSB \endcode Some things to note are that the VRTRasterBand has a subClass specifier of "VRTRawRasterBand". Also, the VRTRawRasterBand contains a number of previously unseen elements but no "source" information. VRTRawRasterBands may never have sources (i.e. SimpleSource), but should contain the following elements in addition to all the normal "metadata" elements previously described which are still supported. A few other notes: Another example, in this case a 400x300 RGB pixel interleaved image. \code Red rgb.raw 0 3 1200 Green rgb.raw 1 3 1200 Blue rgb.raw 2 3 1200 \endcode \section gdal_vrttut_creation Programmatic Creation of VRT Datasets The VRT driver supports several methods of creating VRT datasets. As of GDAL 1.2.0 the vrtdataset.h include file should be installed with the core GDAL include files, allowing direct access to the VRT classes. However, even without that most capabilities remain available through standard GDAL interfaces.

To create a VRT dataset that is a clone of an existing dataset use the CreateCopy() method. For example to clone utm.tif into a wrk.vrt file in C++ the following could be used: \code GDALDriver *poDriver = (GDALDriver *) GDALGetDriverByName( "VRT" ); GDALDataset *poSrcDS, *poVRTDS; poSrcDS = (GDALDataset *) GDALOpenShared( "utm.tif", GA_ReadOnly ); poVRTDS = poDriver->CreateCopy( "wrk.vrt", poSrcDS, FALSE, NULL, NULL, NULL ); GDALClose((GDALDatasetH) poVRTDS); GDALClose((GDALDatasetH) poSrcDS); \endcode Note the use of GDALOpenShared() when opening the source dataset. It is advised to use GDALOpenShared() in this situation so that you are able to release the explicit reference to it before closing the VRT dataset itself. In other words, in the previous example, you could also invert the 2 last lines, whereas if you open the source dataset with GDALOpen(), you'd need to close the VRT dataset before closing the source dataset. To create a virtual copy of a dataset with some attributes added or changed such as metadata or coordinate system that are often hard to change on other formats, you might do the following. In this case, the virtual dataset is created "in memory" only by virtual of creating it with an empty filename, and then used as a modified source to pass to a CreateCopy() written out in TIFF format. \code poVRTDS = poDriver->CreateCopy( "", poSrcDS, FALSE, NULL, NULL, NULL ); poVRTDS->SetMetadataItem( "SourceAgency", "United States Geological Survey"); poVRTDS->SetMetadataItem( "SourceDate", "July 21, 2003" ); poVRTDS->GetRasterBand( 1 )->SetNoDataValue( -999.0 ); GDALDriver *poTIFFDriver = (GDALDriver *) GDALGetDriverByName( "GTiff" ); GDALDataset *poTiffDS; poTiffDS = poTIFFDriver->CreateCopy( "wrk.tif", poVRTDS, FALSE, NULL, NULL, NULL ); GDALClose((GDALDatasetH) poTiffDS); \endcode In the above example the nodata value is set as -999. You can set the HideNoDataValue element in the VRT dataset's band using SetMetadataItem() on that band. \code poVRTDS->GetRasterBand( 1 )->SetMetadataItem( "HideNoDataValue" , "1" ); \endcode In this example a virtual dataset is created with the Create() method, and adding bands and sources programmatically, but still via the "generic" API. A special attribute of VRT datasets is that sources can be added to the VRTRasterBand (but not to VRTRawRasterBand) by passing the XML describing the source into SetMetadata() on the special domain target "new_vrt_sources". The domain target "vrt_sources" may also be used, in which case any existing sources will be discarded before adding the new ones. In this example we construct a simple averaging filter source instead of using the simple source. \code // construct XML for simple 3x3 average filter kernel source. const char *pszFilterSourceXML = "" " utm.tif1" " " " 3" " 0.111 0.111 0.111 0.111 0.111 0.111 0.111 0.111 0.111" " " ""; // Create the virtual dataset. poVRTDS = poDriver->Create( "", 512, 512, 1, GDT_Byte, NULL ); poVRTDS->GetRasterBand(1)->SetMetadataItem("source_0",pszFilterSourceXML, "new_vrt_sources"); \endcode A more general form of this that will produce a 3x3 average filtered clone of any input datasource might look like the following. In this case we deliberately set the filtered datasource as in the "vrt_sources" domain to override the SimpleSource created by the CreateCopy() method. The fact that we used CreateCopy() ensures that all the other metadata, georeferencing and so forth is preserved from the source dataset ... the only thing we are changing is the data source for each band. \code int nBand; GDALDriver *poDriver = (GDALDriver *) GDALGetDriverByName( "VRT" ); GDALDataset *poSrcDS, *poVRTDS; poSrcDS = (GDALDataset *) GDALOpenShared( pszSourceFilename, GA_ReadOnly ); poVRTDS = poDriver->CreateCopy( "", poSrcDS, FALSE, NULL, NULL, NULL ); for( nBand = 1; nBand <= poVRTDS->GetRasterCount(); nBand++ ) { char szFilterSourceXML[10000]; GDALRasterBand *poBand = poVRTDS->GetRasterBand( nBand ); sprintf( szFilterSourceXML, "" " %s%d" " " " 3" " 0.111 0.111 0.111 0.111 0.111 0.111 0.111 0.111 0.111" " " "", pszSourceFilename, nBand ); poBand->SetMetadataItem( "source_0", szFilterSourceXML, "vrt_sources" ); } \endcode The VRTDataset class is one of the few dataset implementations that supports the AddBand() method. The options passed to the AddBand() method can be used to control the type of the band created (VRTRasterBand, VRTRawRasterBand, VRTDerivedRasterBand), and in the case of the VRTRawRasterBand to set its various parameters. For standard VRTRasterBand, sources should be specified with the above SetMetadata() / SetMetadataItem() examples. \code GDALDriver *poDriver = (GDALDriver *) GDALGetDriverByName( "VRT" ); GDALDataset *poVRTDS; poVRTDS = poDriver->Create( "out.vrt", 512, 512, 0, GDT_Byte, NULL ); char** papszOptions = NULL; papszOptions = CSLAddNameValue(papszOptions, "subclass", "VRTRawRasterBand"); // if not specified, default to VRTRasterBand papszOptions = CSLAddNameValue(papszOptions, "SourceFilename", "src.tif"); // mandatory papszOptions = CSLAddNameValue(papszOptions, "ImageOffset", "156"); // optional. default = 0 papszOptions = CSLAddNameValue(papszOptions, "PixelOffset", "2"); // optional. default = size of band type papszOptions = CSLAddNameValue(papszOptions, "LineOffset", "1024"); // optional. default = size of band type * width papszOptions = CSLAddNameValue(papszOptions, "ByteOrder", "LSB"); // optional. default = machine order papszOptions = CSLAddNameValue(papszOptions, "relativeToVRT", "true"); // optional. default = false poVRTDS->AddBand(GDT_Byte, papszOptions); CSLDestroy(papszOptions); delete poVRTDS; \endcode \section gdal_vrttut_derived_c Using Derived Bands (with pixel functions in C/C++) A specialized type of band is a 'derived' band which derives its pixel information from its source bands. With this type of band you must also specify a pixel function, which has the responsibility of generating the output raster. Pixel functions are created by an application and then registered with GDAL using a unique key. Using derived bands you can create VRT datasets that manipulate bands on the fly without having to create new band files on disk. For example, you might want to generate a band using four source bands from a nine band input dataset (x0, x3, x4, and x8): \code band_value = sqrt((x3*x3+x4*x4)/(x0*x8)); \endcode You could write the pixel function to compute this value and then register it with GDAL with the name "MyFirstFunction". Then, the following VRT XML could be used to display this derived band: \code Magnitude MyFirstFunction nine_band.dat 1 nine_band.dat 4 nine_band.dat 5 nine_band.dat 9 \endcode In addition to the subclass specification (VRTDerivedRasterBand) and the PixelFunctionType value, there is another new parameter that can come in handy: SourceTransferType. Typically the source rasters are obtained using the data type of the derived band. There might be times, however, when you want the pixel function to have access to higher resolution source data than the data type being generated. For example, you might have a derived band of type "Float", which takes a single source of type "CFloat32" or "CFloat64", and returns the imaginary portion. To accomplish this, set the SourceTransferType to "CFloat64". Otherwise the source would be converted to "Float" prior to calling the pixel function, and the imaginary portion would be lost. \code Magnitude MyFirstFunction CFloat64 ... \endcode \subsection gdal_vrttut_derived_default_c_pixel_functions Default Pixel Functions Starting with GDAL 2.2, GDAL provides a set of default pixel functions that can be used without writing new code:

\subsection gdal_vrttut_derived_c_pixel_functions Writing Pixel Functions To register this function with GDAL (prior to accessing any VRT datasets with derived bands that use this function), an application calls GDALAddDerivedBandPixelFunc with a key and a GDALDerivedPixelFunc: \code GDALAddDerivedBandPixelFunc("MyFirstFunction", TestFunction); \endcode A good time to do this is at the beginning of an application when the GDAL drivers are registered. GDALDerivedPixelFunc is defined with a signature similar to IRasterIO: @param papoSources A pointer to packed rasters; one per source. The datatype of all will be the same, specified in the eSrcType parameter. @param nSources The number of source rasters. @param pData The buffer into which the data should be read, or from which it should be written. This buffer must contain at least nBufXSize * nBufYSize words of type eBufType. It is organized in left to right, top to bottom pixel order. Spacing is controlled by the nPixelSpace, and nLineSpace parameters. @param nBufXSize The width of the buffer image into which the desired region is to be read, or from which it is to be written. @param nBufYSize The height of the buffer image into which the desired region is to be read, or from which it is to be written. @param eSrcType The type of the pixel values in the papoSources raster array. @param eBufType The type of the pixel values that the pixel function must generate in the pData data buffer. @param nPixelSpace The byte offset from the start of one pixel value in pData to the start of the next pixel value within a scanline. If defaulted (0) the size of the datatype eBufType is used. @param nLineSpace The byte offset from the start of one scanline in pData to the start of the next. @return CE_Failure on failure, otherwise CE_None. \code typedef CPLErr (*GDALDerivedPixelFunc)(void **papoSources, int nSources, void *pData, int nXSize, int nYSize, GDALDataType eSrcType, GDALDataType eBufType, int nPixelSpace, int nLineSpace); \endcode The following is an implementation of the pixel function: \code #include "gdal.h" CPLErr TestFunction(void **papoSources, int nSources, void *pData, int nXSize, int nYSize, GDALDataType eSrcType, GDALDataType eBufType, int nPixelSpace, int nLineSpace) { int ii, iLine, iCol; double pix_val; double x0, x3, x4, x8; // ---- Init ---- if (nSources != 4) return CE_Failure; // ---- Set pixels ---- for( iLine = 0; iLine < nYSize; iLine++ ) { for( iCol = 0; iCol < nXSize; iCol++ ) { ii = iLine * nXSize + iCol; /* Source raster pixels may be obtained with SRCVAL macro */ x0 = SRCVAL(papoSources[0], eSrcType, ii); x3 = SRCVAL(papoSources[1], eSrcType, ii); x4 = SRCVAL(papoSources[2], eSrcType, ii); x8 = SRCVAL(papoSources[3], eSrcType, ii); pix_val = sqrt((x3*x3+x4*x4)/(x0*x8)); GDALCopyWords(&pix_val, GDT_Float64, 0, ((GByte *)pData) + nLineSpace * iLine + iCol * nPixelSpace, eBufType, nPixelSpace, 1); } } // ---- Return success ---- return CE_None; } \endcode \section gdal_vrttut_derived_python Using Derived Bands (with pixel functions in Python) Starting with GDAL 2.2, in addition to pixel functions written in C/C++ as documented in the \ref gdal_vrttut_derived_c section, it is possible to use pixel functions written in Python. Both CPython and NumPy are requirements at run-time. The subelements for VRTRasterBand (whose subclass specification must be set to VRTDerivedRasterBand) are : The signature of the Python pixel function must have the following arguments:
  1. in_ar: list of input NumPy arrays (one NumPy array for each source)
  2. out_ar: output NumPy array to fill. The array is initialized at the right dimensions and with the VRTRasterBand.dataType.
  3. xoff: pixel offset to the top left corner of the accessed region of the band. Generally not needed except if the processing depends on the pixel position in the raster.
  4. yoff line offset to the top left corner of the accessed region of the band. Generally not needed.
  5. xsize: width of the region of the accessed region of the band. Can be used together with out_ar.shape[1] to determine the horizontal resampling ratio of the request.
  6. ysize: height of the region of the accessed region of the band. Can be used together with out_ar.shape[0] to determine the vertical resampling ratio of the request.
  7. raster_xsize: total with of the raster band. Generally not needed.
  8. raster_ysize: total with of the raster band. Generally not needed.
  9. buf_radius: radius of the buffer (in pixels) added to the left, right, top and bottom of in_ar / out_ar. This is the value of the optional BufferRadius element that can be set so that the original pixel request is extended by a given amount of pixels.
  10. gt: geotransform. Array of 6 double values.
  11. kwargs: dictionary with user arguments defined in PixelFunctionArguments
\subsection gdal_vrttut_derived_python_examples Examples