ngpython bindings from cvs do not compile on amd64/python2.5

[schut@…]

ngpython bindings from cvs do not compile on amd64/python2.5

On my pc, probably because it's an amd64 and maybe because I'm running python2.5, I get the following error during make install:

running build_ext
building '_gdal' extension
creating build/temp.linux-x86_64-2.5
x86_64-pc-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr -I/usr/lib64/python2.5/site-packages/numpy/core/include -I/usr/include/python2.5 -c gdal_wrap.cpp -o build/temp.linux-x86_64-2.5/gdal_wrap.o
gdal_wrap.cpp: In function 'int SWIG_Python_ConvertFunctionPtr(PyObject*, void**, swig_type_info*)':
gdal_wrap.cpp:2051: error: invalid conversion from 'const char*' to 'char*'
gdal_wrap.cpp: In function 'int SWIG_AsCharPtrAndSize(PyObject*, char**, size_t*, int*)':
gdal_wrap.cpp:2657: error: cannot convert 'int*' to 'Py_ssize_t*' for argument '3' to 'int PyString_AsStringAndSize(PyObject*, char**, Py_ssize_t*)'
gdal_wrap.cpp: In function 'PyObject* _wrap_Dataset_WriteRaster(PyObject*, PyObject*, PyObject*)':
gdal_wrap.cpp:7603: error: cannot convert 'int*' to 'Py_ssize_t*' for argument '3' to 'int PyString_AsStringAndSize(PyObject*, char**, Py_ssize_t*)'
gdal_wrap.cpp: In function 'PyObject* _wrap_Band_WriteRaster(PyObject*, PyObject*, PyObject*)':
gdal_wrap.cpp:9168: error: cannot convert 'int*' to 'Py_ssize_t*' for argument '3' to 'int PyString_AsStringAndSize(PyObject*, char**, Py_ssize_t*)'
gdal_wrap.cpp: In function 'void SWIG_Python_FixMethods(PyMethodDef*, swig_const_info*, swig_type_info**, swig_type_info**)':
gdal_wrap.cpp:11167: error: invalid conversion from 'const char*' to 'char*'
error: command 'x86_64-pc-linux-gnu-gcc' failed with exit status 1

[warmerdam]

I have confirmed that the swig generated bindings are not Python 2.5
compatible.  Two main problems:

1)  ml_doc changed to "const char *" from "const char*" so lots of use of it
needs to be handled as const char *. 

2) PyString_AsStringAndSize()'s last argument is now Py_ssize_t *, not int *
as it was in 2.4, and at least on 64bit platforms this is actually a different
size.  

I prepared hacked bindings that work on Python 2.5, but these modifications
are not in CVS and not part of the normal process.  So we need to keep this
bug report open till the issue is properly dealh with.

  http://www.gdal.org/tmp/py25ng.tar.gz




  

[warmerdam]

Patch for 2.5 support.

[hobu]

This is a swig problem. I believe it has been fixed in later versions, but we haven't yet gone past 1.3.31. We would have to kick everyone forward to a new version of swig, which causes churn. Pushing this off to look at it again for 1.5.

[vincentschut]

This might not be working for python2.4 users, but for me simply changing line 442 of swig/include/python/typemaps_python.i to use Py_ssize_t* instead of int (so it looks like this: %typemap(in,numinputs=1) (Py_ssize_t *nLen, char *pBuf ) ) fixes this. But probably a python2.4 safe alternative should be found? BTW this is using swig from svn (1.3.32), but I suppose it also works with gdal's current 1.3.31.

[hobu]

#if (PY_VERSION_HEX < 0x02050000)
typedef int Py_ssize_t;
#endif

Will it work to put this after typedef int FALSE_IS_ERR around line 67 of gdal.i?

[hobu]

Replying to hobu:

#if (PY_VERSION_HEX < 0x02050000)
typedef int Py_ssize_t;
#endif

Will it work to put this after typedef int FALSE_IS_ERR around line 67 of gdal.i?

Test should go the other direction.

#if (PY_VERSION_HEX >= 0x02050000)
typedef int Py_ssize_t;
#endif

[hobu]

I think I've now fixed this in r11372 and r11373 so it does the right thing for 2.5 without interfering with <2.4.

[hobu]

karson on #gdal confirmed that it's compiling on Python 2.5.1 now. Closing...

[hobu]

re-opening until backported to 1.4 branch. Currently only fixed in trunk

[hobu]

backported in r11377

[vincentschut]

reopening. I still get the same error (using a fresh svn checkout):

building '_gdal' extension creating build/temp.linux-x86_64-2.5 x86_64-pc-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr -I/usr/lib64/python2.5/site-packages/numpy/core/include -I/usr/include/python2.5 -c gdal_wrap.cpp -o build/temp.linux-x86_64-2.5/gdal_wrap.o -g gdal_wrap.cpp: In function 'PyObject* _wrap_Dataset_WriteRaster(PyObject*, PyObject*, PyObject*)': gdal_wrap.cpp:7608: error: cannot convert 'int*' to 'Py_ssize_t*' for argument '3' to 'int PyString_AsStringAndSize(PyObject*, char, Py_ssize_t*)' gdal_wrap.cpp: In function 'PyObject* _wrap_Band_WriteRaster(PyObject*, PyObject*, PyObject*)': gdal_wrap.cpp:9149: error: cannot convert 'int*' to 'Py_ssize_t*' for argument '3' to 'int PyString_AsStringAndSize(PyObject*, char, Py_ssize_t*)' error: command 'x86_64-pc-linux-gnu-gcc' failed with exit status 1 make[2]: * [install] Error 1 make[2]: Leaving directory `/usr/local/src/gdal/swig/python' make[1]: * [install] Error 2 make[1]: Leaving directory `/usr/local/src/gdal/swig' make: * [install-actions] Error 2

Did I miss something?

[vincentschut]

In addition to my last note, this is with amd64 linux, which is probably the problem. I guess the python2.5 thing was fixed, but not in combination with 64-bitness...

[vincentschut]

[sorry for this conversation to self in multiple episodes...] hmm after make verclean && make generate build install all seems fine... Don't know why? Because of my own amd64/newer swig?

[hobu]

Replying to vincentschut:

[sorry for this conversation to self in multiple episodes...] hmm after make verclean && make generate build install all seems fine... Don't know why? Because of my own amd64/newer swig?

Only the SWIG interface file was committed, not the wrappers is probably the reason. I have updated them in r11385.

[warmerdam]

There are some serious problems with the previous attempt by hobu to use #ifdef's in the .i file since swig still screws it up. But I've discovered that swig 1.3.31 predefines Py_ssize_t so we can just use it directly. I've changed the typemap to this in trunk and 1.4 branch, regenerated the bindings and tested with Python 2.5 and Python 2.4 on AMD64 (really only tested in trunk). This worked, and I have committed all the changes including the updated gdal_wrap.cpp in both places.

%typemap(in,numinputs=1) (int nLen, char *pBuf )
{
  /* %typemap(in,numinputs=1) (int nLen, char *pBuf ) */
  PyString_AsStringAndSize($input, &$2, ( Py_ssize_t*)&$1 );
}

Vincent,

If it is convenient to re-test the svn-stable nightly snapshot tomorrow or some time after that I'd appreciate it.

[vincentschut]

I have just re-tested with svn trunk, and at compilation and install phases went OK for me (python2.5, amd64, linux), so I suppose we can now at last close this issue.