GDALRasterPolygonEnumerator does not merge polygons correctly when nConnectedness = 8

[ctoney]

When nConnectedness = 8 (diagonal pixels considered connected), GDALRasterPolygonEnumerator does not merge some diagonally connected pixels into the same polygon.

The attached graphic shows two examples of connected pixels that were not merged into a single polygon. In the graphic, yellow and blue pixels have the same value and should be enumerated as polygons when connected diagonally or along sides. All other pixels are masked out from polygon enumeration.

The logic in GDALRasterPolygonEnumerator::ProcessLine is incomplete for nConnectedness = 8. Some additional checks for diagonal connectedness are needed with calls to MergePolygon. The attached file gdalrasterpolygonenumerator_fix.cpp contains the additional checks in ProcessLine - all changes are in lines 188-246. The fixed code has been tested on several examples and seems to work on all possible pixel configurations with diagonal connectedness.

Similar changes are probably needed in gdalrasterfpolygonenumerator.cpp (the version using float buffers).

[Kyle Shannon]

Chris, what version are you using?

[Kyle Shannon]

patch against trunk r24372

[Kyle Shannon]

Added patch instead of full file.

[warmerdam]

It would be really nice to have a small test file that demonstrates the problem so we could extend the test suite. See samples in autotest/alg/

I also imagine this needs to be applied to the new floating point copy of the polygonization algorithm.

[warmerdam]

I have applied the change in trunk (r24379) and 1.9 (r24380) to the integer and floating point versions of the algorithm. The floating point side is untested.

[ctoney]

I've attached a small test file that demonstrates the problem of some diagonally connected polygons not merged. Pixel values are 6 or nodata.