gdalwarp fallback to nearest neighbor sampling

[roettger]

When trying to warp a huge file (> ca. 1GB uncompressed) with gdalwarp, it seams that it tries to break it down into tiles and then performs the warp operation on each of the tiles. Those tiles seem to break the continuity of the downsampling operator.

In my particular test case I tried to downsample a huge noisy image by a factor of ca. 10. The resulting image looks fine and the noise is smoothed out except for a 2 pixel frame at the tile boundaries. In that 2 pixel frame the imagery is still very noisy, so that I suspect that gdalwarp falls back from the lanczos or cubicspline operator to nearest neighbor interpolation. This produces very ugly looking boundaries, which make gdalwarp useless for downsampling large noisy files.

I also tried gdaladdo to generate a comparable downsampled overview (.ovr extension) with the average operator. That worked fine as opposed to gdalwarp, but this is not very convenient, because I need to make the original file read-only to produce the .ovr and then I have to rename the .ovr to .tif.

the command line for downsampling the huge noisy image with a resolution of ca. 10cm was:

gdalwarp -tr 1 1 -r cubicspline <source.tif> <output.tif>

On a side note: On my new MacBookPro the support for GDAL --with-python seems to be broken. It appears that the Python setuptools screw up somewhere deep inside of building the swig/python osgeo._gdal extension. I tried to upgrade to the latest gdal-1.5.3 and python 2.6, unfortunately to no avail.

I did the following on MacOS X Leopard MacBook Pro:

./configure --with-python make

error excerpt:

... (cd swig; make build)

for dir in python ; do (cd $dir; make build)

exit; done

python setup.py build numpy include /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/numpy/core/include running build running build_py creating build creating build/lib.macosx-10.5-i386-2.5 copying gdal.py -> build/lib.macosx-10.5-i386-2.5 copying ogr.py -> build/lib.macosx-10.5-i386-2.5 copying osr.py -> build/lib.macosx-10.5-i386-2.5 copying gdalconst.py -> build/lib.macosx-10.5-i386-2.5 copying gdalnumeric.py -> build/lib.macosx-10.5-i386-2.5 creating build/lib.macosx-10.5-i386-2.5/osgeo copying osgeo/init.py -> build/lib.macosx-10.5-i386-2.5/osgeo copying osgeo/gdal.py -> build/lib.macosx-10.5-i386-2.5/osgeo copying osgeo/gdal_array.py -> build/lib.macosx-10.5-i386-2.5/osgeo copying osgeo/gdalconst.py -> build/lib.macosx-10.5-i386-2.5/osgeo copying osgeo/gdalnumeric.py -> build/lib.macosx-10.5-i386-2.5/osgeo copying osgeo/ogr.py -> build/lib.macosx-10.5-i386-2.5/osgeo copying osgeo/osr.py -> build/lib.macosx-10.5-i386-2.5/osgeo running build_ext Could not run gdal-config!!!! building 'osgeo._gdal' extension C compiler: /bin/sh /Users/roettger/mytmp/gdal-1.5.3/libtool --mode=compile --tag=CC cc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -Os -Wall -Wstrict-prototypes -DMACOSX -I/usr/include/ffi -DENABLE_DTRACE -arch i386 -arch ppc -pipe

creating build/temp.macosx-10.5-i386-2.5 creating build/temp.macosx-10.5-i386-2.5/extensions compile options: '-I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -I/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/numpy/core/include -c' sh: extensions/gdal_wrap.cpp libtool: compile: cc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -Os -Wall -Wstrict-prototypes -DMACOSX -I/usr/include/ffi -DENABLE_DTRACE -arch i386 -arch ppc -pipe -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -I/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/numpy/core/include -c extensions/gdal_wrap.cpp -fno-common -DPIC -o build/temp.macosx-10.5-i386-2.5/extensions/.libs/gdal_wrap.o cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++ cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++ extensions/gdal_wrap.cpp: In function 'PyObject* _wrap_Dataset_WriteRaster(PyObject*, PyObject*, PyObject*)': extensions/gdal_wrap.cpp:8527: warning: control may reach end of non-void function 'CPLErr GDALDatasetShadow_WriteRaster(GDALDatasetShadow*, int, int, int, int, int, char*, int*, int*, GDALDataType*, int, int*)' being inlined extensions/gdal_wrap.cpp: In function 'PyObject* _wrap_Dataset_ReadRaster(PyObject*, PyObject*, PyObject*)': extensions/gdal_wrap.cpp:8702: warning: control may reach end of non-void function 'CPLErr GDALDatasetShadow_ReadRaster(GDALDatasetShadow*, int, int, int, int, int*, char, int*, int*, GDALDataType*, int, int*)' being inlined extensions/gdal_wrap.cpp: In function 'PyObject* _wrap_Dataset_WriteRaster(PyObject*, PyObject*, PyObject*)': extensions/gdal_wrap.cpp:8527: warning: control may reach end of non-void function 'CPLErr GDALDatasetShadow_WriteRaster(GDALDatasetShadow*, int, int, int, int, int, char*, int*, int*, GDALDataType*, int, int*)' being inlined extensions/gdal_wrap.cpp: In function 'PyObject* _wrap_Dataset_ReadRaster(PyObject*, PyObject*, PyObject*)': extensions/gdal_wrap.cpp:8702: warning: control may reach end of non-void function 'CPLErr GDALDatasetShadow_ReadRaster(GDALDatasetShadow*, int, int, int, int, int*, char, int*, int*, GDALDataType*, int, int*)' being inlined extensions/gdal_wrap.cpp: In function 'PyObject* _wrap_RasterAttributeTable_SetValueAsString(PyObject*, PyObject*)': extensions/gdal_wrap.cpp:11283: warning: 'str4' may be used uninitialized in this function extensions/gdal_wrap.cpp: In function 'PyObject* _wrap_RasterAttributeTable_SetValueAsString(PyObject*, PyObject*)': extensions/gdal_wrap.cpp:11283: warning: 'str4' may be used uninitialized in this function libtool: compile: cc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -Os -Wall -Wstrict-prototypes -DMACOSX -I/usr/include/ffi -DENABLE_DTRACE -arch i386 -arch ppc -pipe -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -I/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/numpy/core/include -c extensions/gdal_wrap.cpp -o build/temp.macosx-10.5-i386-2.5/extensions/gdal_wrap.o >/dev/null 2>&1 /bin/sh -Wl,-F. -bundle -undefined dynamic_lookup -arch i386 -arch ppc build/temp.macosx-10.5-i386-2.5/extensions/gdal_wrap.o -L../../.libs -L../../ -L/usr/local/lib -L/usr/lib -lgdal -o build/lib.macosx-10.5-i386-2.5/osgeo/_gdal.so /bin/sh: -W: invalid option Usage: /bin/sh [GNU long option] [option] ...

/bin/sh [GNU long option] [option] script-file ...

GNU long options:

--debug --debugger --dump-po-strings --dump-strings --help --init-file --login --noediting --noprofile --norc --posix --protected --rcfile --restricted --verbose --version --wordexp

Shell options:

-irsD or -c command or -O shopt_option (invocation only) -abefhkmnptuvxBCHP or -o option

/bin/sh: -W: invalid option Usage: /bin/sh [GNU long option] [option] ...

/bin/sh [GNU long option] [option] script-file ...

GNU long options:

--debug --debugger --dump-po-strings --dump-strings --help --init-file --login --noediting --noprofile --norc --posix --protected --rcfile --restricted --verbose --version --wordexp

Shell options:

-irsD or -c command or -O shopt_option (invocation only) -abefhkmnptuvxBCHP or -o option

error: Command "/bin/sh -Wl,-F. -bundle -undefined dynamic_lookup -arch i386 -arch ppc build/temp.macosx-10.5-i386-2.5/extensions/gdal_wrap.o -L../../.libs -L../../ -L/usr/local/lib -L/usr/lib -lgdal -o build/lib.macosx-10.5-i386-2.5/osgeo/_gdal.so" failed with exit status 2 make[2]: * [build] Error 1 make[1]: * [build] Error 2 make: * [swig-modules] Error 2

It seems that "/bin/sh -Wl,-F. -bundle" is called with the proper libtool linker command missing after /bin/sh.

I am not sure whether or not this is a Python or GDAL related problem. Would someone please forward that to the appropriate bug tracker.

Thanks, Stefan

[Even Rouault]

Have you tried running against a recent GDAL version from SVN trunk ? A lot of improvements have been commited in that area recently that may solve the issue you observe on the boundaries of the pieces of the image that is warped.

[roettger]

Replying to rouault:

Have you tried running against a recent GDAL version from SVN trunk ? A lot of improvements have been commited in that area recently that may solve the issue you observe on the boundaries of the pieces of the image that is warped.

Thanks for the hint, I am about to check out the latest gdal version from svn and give it another try...

[roettger]

I checked the latest gdal version from svn. That fixes the sampling issues at the tile boundaries.

The latest version, however, still does not compile with "--with-python" on my MacBook Pro with MacOS X Leopard.

[Even Rouault]

for the python bindings subject, I can't really say what's going wrong for you. You can open a ticketn assign PythonBindings for the component and specify svn-trunk as the version