Changes between Initial Version and Version 1 of PythonGotchas


Ignore:
Timestamp:
Sep 20, 2010 8:00:22 AM (9 years ago)
Author:
jjr8
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PythonGotchas

    v1 v1  
     1= Gotchas in the GDAL and OGR Python Bindings =
     2
     3This page lists aspects of GDAL's and OGR's Python bindings that may catch Python programmers by surprise. If you find something new, feel free to add it to the list, but consider discussing it on the [http://www.osgeo.org/mailman/listinfo/gdal-dev/ gdal-dev mailing list] first, to make sure you fully understand the issue and that others agree that it is unexpected, "non-Pythonic", or something that would catch many Python programmers by surprise. Be sure to reference email threads, Trac tickets, and other sources of additional information.
     4
     5This list is not the place to report bugs. If you believe something is a bug, please [http://trac.osgeo.org/gdal/newticket open a ticket] and report the problem to gdal-dev. Then consider listing it here if it is something related to Python specifically. Do not list it here if it relates to GDAL or OGR generally, and not the Python bindings specifically.
     6
     7Not all items listed here are bugs. Some of these are just how GDAL and OGR work and will never be "fixed" per se. If you don't like how something works and think it should be changed, feel free to discuss it on gdal-dev and see what can be done.
     8
     9= Gotchas that are by design =
     10
     11These are unexpected behaviors that are not considered by the GDAL and OGR teams to be bugs and are unlikely to be changed due to effort required, the need to maintain backward compatibility, etc.
     12
     13=== Python bindings do not raise exceptions unless you explicitly call {{{UseExceptions()}}} ===
     14
     15By default, the GDAL and OGR Python bindings do not raise exceptions when errors occur. Instead they return an error value such as {{{None}}} and write an error message to {{{sys.stdout}}}. For example, when you try to open a non-existing dataset with GDAL:
     16
     17{{{
     18#!python
     19>>> from osgeo import gdal
     20>>> gdal.Open('C:\\foo.img')
     21ERROR 4: `C:\foo.img' does not exist in the file system,
     22and is not recognised as a supported dataset name.
     23
     24>>>
     25}}}
     26
     27In Python, it is traditional to report errors by raising exceptions. You can enable this behavior in GDAL and OGR by calling the {{{UseExceptions()}}} function:
     28
     29{{{
     30#!python
     31>>> from osgeo import gdal
     32>>> gdal.UseExceptions()        # Enable exceptions
     33>>> gdal.Open('C:\\foo.img')
     34Traceback (most recent call last):
     35  File "<stdin>", line 1, in <module>
     36RuntimeError: `C:\foo.img' does not exist in the file system,
     37and is not recognised as a supported dataset name.
     38
     39>>>
     40}}}
     41
     42The GDAL team acknowledges that Python programmers expect exceptions to be enabled by default, but says that exceptions are disabled by default to [http://lists.osgeo.org/pipermail/gdal-dev/2010-September/026031.html preserve backward compatibility].
     43
     44= Gotchas fixed in GDAL 1.8.0 =
     45
     46These are bugs that were fixed or designs that were changed in GDAL 1.8.0 and later. If you use an older version, watch out for these.
     47
     48=== {{{gdal.ErrorReset()}}} must be called after an error occurs, or it will keep happening ===
     49
     50In this example, we use OGR to create a geometry object using valid WKT. Then we try some invalid WKT, which is expected to fail, and then try the valid WKT again but it fails. Other OGR functions will also fail.
     51
     52{{{
     53#!python
     54>>> from osgeo import ogr
     55>>> ogr.UseExceptions()
     56>>> ogr.CreateGeometryFromWkt('POINT(1 2)')          # Create a point using valid WKT
     57<osgeo.ogr.Geometry; proxy of <Swig Object of type 'OGRGeometryShadow *' at 0x244b658> >
     58>>> ogr.CreateGeometryFromWkt('blah blah blah')      # Now try to create one using invalid WKT
     59Traceback (most recent call last):
     60  File "<stdin>", line 1, in <module>
     61  File "C:\Python25\lib\site-packages\osgeo\ogr.py", line 2885, in CreateGeometryFromWkt
     62    return _ogr.CreateGeometryFromWkt(*args, **kwargs)
     63RuntimeError: OGR Error: Unsupported geometry type
     64>>> ogr.CreateGeometryFromWkt('POINT(1 2)')          # Now try to to create one using valid WKT again
     65Traceback (most recent call last):
     66  File "<stdin>", line 1, in <module>
     67  File "C:\Python25\lib\site-packages\osgeo\ogr.py", line 2885, in CreateGeometryFromWkt
     68    return _ogr.CreateGeometryFromWkt(*args, **kwargs)
     69RuntimeError: OGR Error: Unsupported geometry type
     70}}}
     71
     72The problem is that OGR and GDAL maintain an internal state variable that tracks whether an error occurred during the last operation, but that this variable is not automatically cleared by OGR or GDAL. You must manually clear it by calling the {{{gdal.ErrorReset()}}} function:
     73
     74{{{
     75#!python
     76>>> ogr.CreateGeometryFromWkt('blah blah blah')
     77Traceback (most recent call last):
     78  File "<stdin>", line 1, in <module>
     79  File "C:\Python25\lib\site-packages\osgeo\ogr.py", line 2885, in CreateGeometryFromWkt
     80    return _ogr.CreateGeometryFromWkt(*args, **kwargs)
     81RuntimeError: OGR Error: Unsupported geometry type
     82>>> from osgeo import gdal
     83>>> gdal.ErrorReset()
     84>>> ogr.CreateGeometryFromWkt('POINT(1 2)')
     85<osgeo.ogr.Geometry; proxy of <Swig Object of type 'OGRGeometryShadow *' at 0x244b7a8> >
     86}}}
     87
     88This function only appears in the GDAL Python bindings, not the OGR Python bindings. Even if you are only using OGR, you must use GDAL to clear the error.
     89
     90This problem is ackowledged by the GDAL team as a bug. Please see #3077.
     91
     92= Gotchas that result from bugs or behaviors of other software =
     93
     94=== Python crashes in GDAL functions when you upgrade or downgrade numpy ===
     95
     96Much of GDAL's Python bindings are implemented in C++. Much of the core of numpy is also implemented in C++. The C++ part of GDAL's Python bindings interacts with the C++ part of numpy through numpy's ABI (application binary interface). This requires GDAL's Python bindings to be compiled using numpy header files that define numpy C++ data structures. Those data structures sometimes change between numpy versions. When this happens, the new version of numpy is not be compatible at the binary level with the old version, and the GDAL Python bindings must be recompiled before they will work with the new verison of numpy. And when they are recompiled, they probably won't work with the old verison.
     97
     98If you obtain a precompiled version of GDAL's Python bindings, such as the Windows packages from http://vbkto.dyndns.org/sdk/, be sure you look up what version of numpy was used to compile them, and install that version of numpy on your machine.
     99
     100=== Python bindings cannot be used successfully from ArcGIS in-process geoprocessing tools (ArcGIS 9.3 and later) ===
     101
     102ArcGIS allows the creation of custom, Python-based geoprocessing tools. Until ArcGIS 10, there was no easy way to read raster data into memory. GDAL provides such a mechanism.
     103
     104Starting with ArcGIS 9.3, geoprocessing tools can either run in the ArcGIS process itself (!ArcCatalog.exe or !ArcMap.exe) or run in a separate python.exe worker process. Unfortunately ArcGIS contains a bug in how it runs in-process tools. Thus, if you use GDAL from an in-process tool, it will run fine the first time but after that it may fail with {{{TypeError}}} exceptions until you restart the ArcGIS process. For example, band.!ReadAsArray() fails with:
     105
     106    {{{TypeError: in method 'BandRasterIONumPy', argument 1 of type 'GDALRasterBandShadow *'}}}
     107
     108This is a bug in ArcGIS. Please see #3672 for complete details and advice on workarounds.