Strange Python dependency to libNCSCnet

[Mateusz Łoskot]

I'm building GDAL with ECW support on Mac OS X 10.4 using ./configure --with-ecw and linking against libecwj2-3.3 binaries.

~/dev/gdal/_svn/trunk/gdal $ otool -L /Users/mloskot/dev/gdal/_svn/trunk/gdal/libgdal.dylib |grep NCS
        /usr/local/lib/libNCSEcw.0.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/local/lib/libNCSCnet.0.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/local/lib/libNCSUtil.0.dylib (compatibility version 1.0.0, current version 1.0.0)

I also request Python bindings using --without-ogpython --with-python.

Everything have worked well for long time. As I remember it worked well also in last Thursday (Now 29). Today, I updated GDAL from SVN and tried to rebuild & test but loading Python bindings fail with strange error:

~/dev/gdal/_svn/trunk/gdal $ python
Python 2.3.5 (#1, Dec  7 2006, 14:50:51) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363) (+4864187)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import gdal
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/Users/mloskot/dev/gdal/_svn/trunk/gdal/swig/python/build/lib.darwin-8.11.1-i386-2.3/gdal.py", line 2, in ?
    from osgeo.gdal import deprecation_warn
  File "/Users/mloskot/dev/gdal/_svn/trunk/gdal/swig/python/build/lib.darwin-8.11.1-i386-2.3/osgeo/gdal.py", line 7, in ?
    import _gdal
ImportError: dlopen(/Users/mloskot/dev/gdal/_svn/trunk/gdal/swig/python/build/lib.darwin-8.11.1-i386-2.3/osgeo/_gdal.so, 2): Symbol not found: _GetControlByID
  Referenced from: /usr/local/lib/libNCSCnet.0.dylib
  Expected in: flat namespace

>>> 

The reported libNCSCnet library belongs to libecwj2-3.3 SDK:

$ nm /usr/local/lib/libNCSCnet.0.dylib |grep ID
         U _GetControlByID
         U _GetControlID
...

Here is where I suck the GDAL Python bindings from:

~/dev/gdal/_svn $ echo $DYLD_LIBRARY_PATH  
/Users/mloskot/dev/gdal/_svn/trunk/gdal

~/dev/gdal/_svn/trunk/gdal $ echo $PYTHONPATH
/Users/mloskot/dev/gdal/_svn/trunk/gdal/swig/python/build/lib.darwin-8.11.1-i386-2.3

~/dev/gdal/_svn/trunk/gdal $ ls swig/python/build/lib.darwin-8.11.1-i386-2.3/osgeo/
__init__.py    _gdalconst.so  _osr.so        gdal_array.py  gdalnumeric.py osr.py
_gdal.so       _ogr.so        gdal.py        gdalconst.py   ogr.py

I'm wondering if it is a bug or something specific to my environment.

However, everything works well when using GDAL without ECW support:

~/dev/gdal/_svn/trunk/gdal $ otool -L /Users/mloskot/dev/gdal/_svn/trunk/gdal/libgdal.dylib | grep NCS
~/dev/gdal/_svn/trunk/gdal 

and I can import GDAL modules with success:

~/dev/gdal/_svn/trunk/gdal $ python
Python 2.3.5 (#1, Dec  7 2006, 14:50:51) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363) (+4864187)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import gdal
/Users/mloskot/dev/gdal/_svn/trunk/gdal/swig/python/build/lib.darwin-8.11.1-i386-2.3/osgeo/gdal.py:81: DeprecationWarning: gdal.py was placed in a namespace, it is now available as osgeo.gdal
  DeprecationWarning)
>>>

[Mateusz Łoskot]

Here is complete set of commands issued by python setup.py build, so you can see linker flags (ECW enabled by ./configure):

~/dev/gdal/_svn/trunk/gdal/swig/python $ make
python setup.py build
running build
running build_py
creating build
creating build/lib.darwin-8.11.1-i386-2.3
copying gdal.py -> build/lib.darwin-8.11.1-i386-2.3
copying ogr.py -> build/lib.darwin-8.11.1-i386-2.3
copying osr.py -> build/lib.darwin-8.11.1-i386-2.3
copying gdalconst.py -> build/lib.darwin-8.11.1-i386-2.3
creating build/lib.darwin-8.11.1-i386-2.3/osgeo
copying osgeo/__init__.py -> build/lib.darwin-8.11.1-i386-2.3/osgeo
copying osgeo/gdal.py -> build/lib.darwin-8.11.1-i386-2.3/osgeo
copying osgeo/gdal_array.py -> build/lib.darwin-8.11.1-i386-2.3/osgeo
copying osgeo/gdalconst.py -> build/lib.darwin-8.11.1-i386-2.3/osgeo
copying osgeo/gdalnumeric.py -> build/lib.darwin-8.11.1-i386-2.3/osgeo
copying osgeo/ogr.py -> build/lib.darwin-8.11.1-i386-2.3/osgeo
copying osgeo/osr.py -> build/lib.darwin-8.11.1-i386-2.3/osgeo
running build_ext
building 'osgeo._gdal' extension
creating build/temp.darwin-8.11.1-i386-2.3
creating build/temp.darwin-8.11.1-i386-2.3/extensions
gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.3/include/python2.3 -I. -I/Users/mloskot/dev/gdal/_svn/trunk/gdal/include -c extensions/gdal_wrap.cpp -o build/temp.darwin-8.11.1-i386-2.3/extensions/gdal_wrap.o
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++
extensions/gdal_wrap.cpp: In function 'PyObject* _wrap_RasterAttributeTable_SetValueAsString(PyObject*, PyObject*)':
extensions/gdal_wrap.cpp:11283: warning: 'str4' may be used uninitialized in this function
extensions/gdal_wrap.cpp: In function 'PyObject* _wrap_Dataset_ReadRaster(PyObject*, PyObject*, PyObject*)':
extensions/gdal_wrap.cpp:8702: warning: control may reach end of non-void function 'CPLErr GDALDatasetShadow_ReadRaster(GDALDatasetShadow*, int, int, int, int, int*, char**, int*, int*, GDALDataType*, int, int*)' being inlined
extensions/gdal_wrap.cpp: In function 'PyObject* _wrap_Dataset_WriteRaster(PyObject*, PyObject*, PyObject*)':
extensions/gdal_wrap.cpp:8527: warning: control may reach end of non-void function 'CPLErr GDALDatasetShadow_WriteRaster(GDALDatasetShadow*, int, int, int, int, int, char*, int*, int*, GDALDataType*, int, int*)' being inlined
c++ -Wl,-F. -Wl,-F. -bundle -undefined dynamic_lookup build/temp.darwin-8.11.1-i386-2.3/extensions/gdal_wrap.o -L../../.libs -L../../ -L/usr/local/lib -L/usr/lib -L/Users/mloskot/dev/gdal/_svn/trunk/gdal/lib -lgdal -o build/lib.darwin-8.11.1-i386-2.3/osgeo/_gdal.so
/usr/bin/ld: warning -L: directory name (../../.libs) does not exist
/usr/bin/ld: warning -L: directory name (/Users/mloskot/dev/gdal/_svn/trunk/gdal/lib) does not exist
building 'osgeo._gdalconst' extension
gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.3/include/python2.3 -I. -I/Users/mloskot/dev/gdal/_svn/trunk/gdal/include -c extensions/gdalconst_wrap.c -o build/temp.darwin-8.11.1-i386-2.3/extensions/gdalconst_wrap.o
gcc -Wl,-F. -Wl,-F. -bundle -undefined dynamic_lookup build/temp.darwin-8.11.1-i386-2.3/extensions/gdalconst_wrap.o -L../../.libs -L../../ -L/usr/local/lib -L/usr/lib -L/Users/mloskot/dev/gdal/_svn/trunk/gdal/lib -lgdal -o build/lib.darwin-8.11.1-i386-2.3/osgeo/_gdalconst.so
/usr/bin/ld: warning -L: directory name (../../.libs) does not exist
/usr/bin/ld: warning -L: directory name (/Users/mloskot/dev/gdal/_svn/trunk/gdal/lib) does not exist
building 'osgeo._osr' extension
gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.3/include/python2.3 -I. -I/Users/mloskot/dev/gdal/_svn/trunk/gdal/include -c extensions/osr_wrap.cpp -o build/temp.darwin-8.11.1-i386-2.3/extensions/osr_wrap.o
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++
c++ -Wl,-F. -Wl,-F. -bundle -undefined dynamic_lookup build/temp.darwin-8.11.1-i386-2.3/extensions/osr_wrap.o -L../../.libs -L../../ -L/usr/local/lib -L/usr/lib -L/Users/mloskot/dev/gdal/_svn/trunk/gdal/lib -lgdal -o build/lib.darwin-8.11.1-i386-2.3/osgeo/_osr.so
/usr/bin/ld: warning -L: directory name (../../.libs) does not exist
/usr/bin/ld: warning -L: directory name (/Users/mloskot/dev/gdal/_svn/trunk/gdal/lib) does not exist
building 'osgeo._ogr' extension
gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.3/include/python2.3 -I. -I/Users/mloskot/dev/gdal/_svn/trunk/gdal/include -c extensions/ogr_wrap.cpp -o build/temp.darwin-8.11.1-i386-2.3/extensions/ogr_wrap.o
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++
extensions/ogr_wrap.cpp: In function 'PyObject* _wrap_Feature_SetField(PyObject*, PyObject*)':
extensions/ogr_wrap.cpp:7089: warning: 'str3' may be used uninitialized in this function
extensions/ogr_wrap.cpp:7149: warning: 'str3' may be used uninitialized in this function
c++ -Wl,-F. -Wl,-F. -bundle -undefined dynamic_lookup build/temp.darwin-8.11.1-i386-2.3/extensions/ogr_wrap.o -L../../.libs -L../../ -L/usr/local/lib -L/usr/lib -L/Users/mloskot/dev/gdal/_svn/trunk/gdal/lib -lgdal -o build/lib.darwin-8.11.1-i386-2.3/osgeo/_ogr.so
/usr/bin/ld: warning -L: directory name (../../.libs) does not exist
/usr/bin/ld: warning -L: directory name (/Users/mloskot/dev/gdal/_svn/trunk/gdal/lib) does not exist
~/dev/gdal/_svn/trunk/gdal/swig/python $

[hobu]

Mateusz,

The linker options of ECW are what's causing the problem. It needs to use the same linking options as the Python bindings:

ldd -Wl,-F. -Wl,-F. -bundle -undefined dynamic_lookup

Also, I've never gotten ECW to actually work on OS X. William may have had some success lately, but I know he's had a lot of trouble too.

Howard