SIGABRT with GEOSDistance_r on Fedora 13, 64 bit

[atrofast]

The following program:

#include <geos_c.h>

int main()
{
        GEOSGeometry *g1, *g2;
        GEOSContextHandle_t handle;
        GEOSWKTReader *wktReader;
        double dist;
        handle = initGEOS_r(NULL, NULL);

        wktReader = GEOSWKTReader_create_r(handle);

        g1 = GEOSWKTReader_read_r(handle, wktReader, "POINT(10 10)");
        g2 = GEOSWKTReader_read_r(handle, wktReader, "POINT(3 6)");

        GEOSDistance_r(handle, g1, g2, &dist);
}

Will cause a SIGABRT with the following backtrace:

(gdb) bt
#0  0x0000003e722329a5 in raise () from /lib64/libc.so.6
#1  0x0000003e72234185 in abort () from /lib64/libc.so.6
#2  0x0000003e7222b935 in __assert_fail () from /lib64/libc.so.6
#3  0x00007ffff7aec673 in geos::geom::GeometryComponentFilter::filter_ro (this=<value optimized out>, geom=<value optimized out>) at GeometryComponentFilter.cpp:35
#4  0x00007ffff7b4e8e5 in getLines (this=0x7fffffffc1e0) at ../../../include/geos/geom/util/LinearComponentExtracter.h:57
#5  geos::operation::distance::DistanceOp::computeFacetDistance (this=0x7fffffffc1e0) at DistanceOp.cpp:389
#6  0x00007ffff7b4ed3d in geos::operation::distance::DistanceOp::distance (this=0x7fffffffc1e0) at DistanceOp.cpp:141
#7  0x00007ffff7b4ee50 in geos::operation::distance::DistanceOp::distance (g0=<value optimized out>, g1=<value optimized out>) at DistanceOp.cpp:61
#8  0x00007ffff7dce78c in GEOSDistance_r (extHandle=0x6024d0, g1=<value optimized out>, g2=<value optimized out>, dist=0x7fffffffc288) at geos_ts_c.cpp:811
#9  0x0000000000400743 in main ()

On Fedora 13 64 bit. I couldn't reproduce it on Arch Linux 64 bit or Fedora 11 32 bit. I did raise this backtrace on the mailing list ( http://lists.osgeo.org/pipermail/geos-devel/2010-August/004933.html) when it was coming from within Ingres but my test case shows it can be triggered independently. It would be interesting to try on Fedora 13 32 bit to see if that makes a difference.

[atrofast]

I found a work around: By default the SVN trunk version compiles with O2 optimization which seems to be the root cause of this bug. If you change CFLAGS/CXXFLAGS to be O1 instead the problem seems to go away.

[rouault]

I experiment the same crash with ScientificLinux? 6 x86_64 (a respin of RHEL 6) and its GCC 4.4.4

[strk]

I've committed a unit test for GEOSDistance with r3442. Works on 64bit with gcc (Ubuntu 4.4.3-4ubuntu5) 4.4.3

I do belive it's a compiler bug you're facing.

[rouault]

Note I've still reproduced the issue with CentOS 6.3 64bit, and the following trick given in #469 solved the issue :

CFLAGS="-m64" CPPFLAGS="-m64" CXXFLAGS="-m64" LDFLAGS="-m64" FFLAGS="-m64" LDFLAGS="-L/usr/lib64/" ./configure