SWIG_MakePtr is undefined in _gdalmodule.so

[Mateusz Łoskot]

Some time ago Paul Van Deusen reported following error when trying to use Python scripts from GDAL:

gdal_merge.py -pct -o temp.tif scAlb.tif vaAlb.tifTraceback (most recent call last):
 File "/usr/local/bin/gdal_merge.py", line 110, in ?
   import gdal
 File "/usr/lib/python2.4/site-packages/gdal.py", line 191, in ?
   import _gdal
ImportError: /usr/lib/python2.4/site-packages/_gdalmodule.so: undefined symbol: SWIG_MakePtr

I confirm the problem occurs in GDAL 1.4.1 but it does not occur in 1.4.0 and current SVN (coming 1.4.2). Strange.

The reason SWIG_MakePtr can not be found is that it's marked as undefined (U) in _gdalmodule.so binary from 1.4.1. The symbol is defined in other versions mentioned above. Here is the proof:

• GDAL 1.4.1

mloskot:~/dev/gdal/release/gdal-1.4.1/pymod$ nm _gdalmodule.so | grep
SWIG_MakePtr
         U SWIG_MakePtr

• GDAL 1.4.0

mloskot:~/dev/gdal/release/gdal-1.4.0/pymod$ nm _gdalmodule.so | grep
SWIG_MakePtr
0000aeb0 t SWIG_MakePtr

• GDAL from current SVN trunk

mloskot:~/dev/gdal/_svn/trunk/gdal/pymod$ nm _gdalmodule.so | grep
SWIG_MakePtr
0000af63 t SWIG_MakePtr 

[Mateusz Łoskot]

Today, Alessandro Frigeri (geoalf) reported the same issue occurring in GDAL package from Debian/unstable:

just got 1.4.1 in debian/unstable, but as I import the python module,
it gives back:
ImportError: /usr/lib/python2.4/site-packages/_gdalmodule.so:
undefined symbol: SWIG_MakePtr.
Recompiling the deb-src fixes it.
Someone already knows about this issue?

[Mateusz Łoskot]

Paul,

I've taken the liberty to add your e-mail to the CC field because I think you may be interested in this report.

[Mateusz Łoskot]

I tested current (today) version of branches/1.4 and it works. The 1.4 branch is pre-1.4.2 version.

The SWIG_MakePtr is defined:

mloskot:~/dev/gdal/_svn/branches/1.4/gdal/pymod$ nm _gdalmodule.so | grep MakePtr
0000af63 t SWIG_MakePtr

and gdal_merge.py works:

mloskot:~/dev/gdal/_svn/branches/1.4/gdal/pymod$ ./gdal_merge.py 
No input files selected.
Usage: gdal_merge.py [-o out_filename] [-of out_format] [-co NAME=VALUE]*
                     [-ps pixelsize_x pixelsize_y] [-separate] [-v] [-pct]
                     [-ul_lr ulx uly lrx lry] [-n nodata_value] [-init value]
                     [-ot datatype] [-createonly] input_files
                     [--help-general]

So, there must be something wrong only in 1.4.1, nothing earlier or later.

[Mateusz Łoskot]

The tags/1.4.1 is affected by the problem:

mloskot:~/dev/gdal/_svn/tags/1.4.1/gdal/pymod$ nm _gdalmodule.so |grep MakePtr
         U SWIG_MakePtr

[Mateusz Łoskot]

I found another interesting difference between gdal-1.4.1 and tags/1.4.1:

• gdal-1.4.1

  mloskot:~/dev/gdal/release/gdal-1.4.1/pymod$ grep SWIG_GLOBAL *
  mloskot:~/dev/gdal/release/gdal-1.4.1/pymod$ 
  

• tags/1.4.1

  mloskot:~/dev/gdal/_svn/tags/1.4.1/gdal/pymod$ grep -n SWIG_GLOBAL *
  gdal_wrap.c:60:#ifdef SWIG_GLOBAL
  gdal_wrap.c:266:#ifdef SWIG_GLOBAL
  mloskot:~/dev/gdal/_svn/tags/1.4.1/gdal/pymod$ 
  

The SWIG_GLOBAL definition controls selection of value of SWIGSTATIC:

#ifdef SWIG_GLOBAL
#ifdef __cplusplus
#define SWIGSTATIC extern "C"
#else
#define SWIGSTATIC
#endif
#endif

#ifndef SWIGSTATIC
#define SWIGSTATIC static
#endif

The SWIGSTATIC specifier is used in SWIG_MakePtr definition.

This difference shows that the content of pymod source code is different in GDAL 1.4.1 and tags/1.4.1, where the latter is considered as the base tree for release package.

• Why they are different?

[Mateusz Łoskot]

Replying to mloskot:

The tags/1.4.1 is affected by the problem:

mloskot:~/dev/gdal/_svn/tags/1.4.1/gdal/pymod$ nm _gdalmodule.so |grep MakePtr
         U SWIG_MakePtr

This is WRONG !

I tried with clean tree of tags/1.4.1 and it works:

...
make[1]: Leaving directory `/home/mloskot/dev/gdal/_svn/tags/1.4.1/gdal/apps'
mloskot:~/dev/gdal/_svn/tags/1.4.1/gdal$ cd pymod/
mloskot:~/dev/gdal/_svn/tags/1.4.1/gdal/pymod$ nm _gdalmodule.so |grep MakePtr
0000af63 t SWIG_MakePtr
mloskot:~/dev/gdal/_svn/tags/1.4.1/gdal/pymod$ echo $PYTHONPATH
/home/mloskot/dev/gdal/_svn/tags/1.4.1/gdal/pymod
mloskot:~/dev/gdal/_svn/tags/1.4.1/gdal/pymod$ ./gdal_merge.py 
No input files selected.
Usage: gdal_merge.py [-o out_filename] [-of out_format] [-co NAME=VALUE]*
                     [-ps pixelsize_x pixelsize_y] [-separate] [-v] [-pct]
                     [-ul_lr ulx uly lrx lry] [-n nodata_value] [-init value]
                     [-ot datatype] [-createonly] input_files
                     [--help-general]

mloskot:~/dev/gdal/_svn/tags/1.4.1/gdal/pymod$

So, in my opinion the bug is only in the release package of gdal-1.4.1.

[Mateusz Łoskot]

This is diff output comparing gdal_wrap.c file from GDAL 1.4.1 release and tags/1.4.1. Why these files are so different?

[Mateusz Łoskot]

Finally, I found the most important difference in gdal_wrap.c file from GDAL 1.4.1 and tags/1.4.1.

The gdal_wrap.c file from the ​gdal-1.4.1 package, does not include definition of the SWIG_MakePtr function.

This function is defined in the gdal_wrap.c file from tags/1.4.1, lines 386 - 409:

SWIGSTATIC 
void SWIG_MakePtr(char *_c, const void *_ptr, char *type) {
  static char _hex[16] =
  {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
   'a', 'b', 'c', 'd', 'e', 'f'};
  unsigned long _p, _s;
  char _result[20], *_r;    /* Note : a 64-bit hex number = 16 digits */
  _r = _result;
  _p = (unsigned long) _ptr;
  if (_p > 0) {
    while (_p > 0) {
      _s = _p & 0xf;
      *(_r++) = _hex[_s];
      _p = _p >> 4;
    }
    *_r = '_';
    while (_r >= _result)
      *(_c++) = *(_r--);
  } else {
    strcpy (_c, "NULL");
  }
  if (_ptr)
    strcpy (_c, type);
}

I'm inclined to judge this is the reason of undefined SWIG_MakePtr in the _gdalmodule.so binary built from gdal-1.4.1 release.

• Why this function is missing from gdal_wrap.c in the gdal-1.4.1 release?

[Mateusz Łoskot]

Frank,

Do you have any thoughts to the investigation I made above? What are we going to do with this report?

[Mateusz Łoskot]

After long and detailed investigation, we've came to the conclusion that this problem is most likely caused by old/unsupported SWIG version used to generate gdal_wrap.c for 1.4.1 release.

We're going to leave this ticket open to remember the coming release 1.4.2 needs to be carefully tested against this problem. If everything works well, then the ticket will be closed as fixed.

[Mateusz Łoskot]

We've not reproduced the problem since a ~month, so I'm closing this bug as fixed for 1.4.2.