Opened 4 years ago

Closed 4 years ago

#4582 closed defect (fixed)

ST_MakeValid: long uncancellable calculation

Reported by: komzpa Owned by: pramsey
Priority: medium Milestone: PostGIS 3.0.3
Component: postgis Version: 2.5.x -- EOL
Keywords: Cc:

Description

PostGIS built for GEOS 3.7 with GEOS 3.8. Same on 3.7 and 2.5.

Polygon is natural disaster extent.

│ POSTGIS="3.0.0 r17983" [EXTENSION] PGSQL="120" GEOS="3.8.0-CAPI-1.13.1 " SFCGAL="1.3.7" PROJ="Rel. 5.2.0, September 15th, 2018" GDAL="GDAL 2.4.2, released 2019/06/28" LIBXML="2.9.4" LIBJSON="0.13.1" LIBPROTOBUF="1.3.1" WAGYU="0.4.3 (Internal)" RASTER │ 

0x00007f436c66b1f6 in geos::geom::GeometryCollection::getCoordinateDimension() const () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
(gdb) bt full
#0  0x00007f436c66b1f6 in geos::geom::GeometryCollection::getCoordinateDimension() const () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
No symbol table info available.
#1  0x00007f436c6dffc0 in geos::operation::overlay::OverlayOp::labelIncompleteNode(geos::geomgraph::Node*, int) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
No symbol table info available.
#2  0x00007f436c6e0165 in geos::operation::overlay::OverlayOp::labelIncompleteNodes() () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
No symbol table info available.
#3  0x00007f436c6e13a2 in geos::operation::overlay::OverlayOp::computeOverlay(geos::operation::overlay::OverlayOp::OpCode) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
No symbol table info available.
#4  0x00007f436c6e16c9 in geos::operation::overlay::OverlayOp::getResultGeometry(geos::operation::overlay::OverlayOp::OpCode) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
No symbol table info available.
#5  0x00007f436c6e1b4c in geos::operation::overlay::OverlayOp::overlayOp(geos::geom::Geometry const*, geos::geom::Geometry const*, geos::operation::overlay::OverlayOp::OpCode) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
No symbol table info available.
#6  0x00007f436c66a0de in std::unique_ptr<geos::geom::Geometry, std::default_delete<geos::geom::Geometry> > geos::geom::BinaryOp<geos::operation::overlay::overlayOp>(geos::geom::Geometry const*, geos::geom::Geometry const*, geos::operation::overlay::overlayOp) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
No symbol table info available.
#7  0x00007f436c667b23 in geos::geom::Geometry::difference(geos::geom::Geometry const*) const () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
No symbol table info available.
#8  0x00007f436c812819 in GEOSDifference_r () from /usr/lib/x86_64-linux-gnu/libgeos_c.so.1
No symbol table info available.
#9  0x00007f436c930e0c in LWGEOM_GEOS_makeValidPolygon (gin=<optimized out>) at lwgeom_geos_clean.c:390
        pi = 0x55bdaf6bcb80
        po = 0x55bdaff114c0
        geos_cut_edges = 0x55bdb05e4fd0
        collapse_points = <optimized out>
        nvgeoms = 0
        gout = <optimized out>
        geos_bound = 0x55bdaefc60a0
        geos_area = <optimized out>
        vgeoms = {0x55bdaece0cd0, 0x55bdaece2ce0, 0x55bdae95b990}
        gout = <optimized out>
        geos_bound = <optimized out>
        geos_cut_edges = <optimized out>
        geos_area = <optimized out>
        collapse_points = <optimized out>
        vgeoms = <optimized out>
        nvgeoms = <optimized out>
        __PRETTY_FUNCTION__ = "LWGEOM_GEOS_makeValidPolygon"
        pi = <optimized out>
        po = <optimized out>
        new_area = <optimized out>
        new_area_bound = <optimized out>
        symdif = <optimized out>
        new_cut_edges = <optimized out>
#10 LWGEOM_GEOS_makeValid (gin=<optimized out>) at lwgeom_geos_clean.c:787
        gout = <optimized out>
        ret_char = <optimized out>
        gout = <optimized out>
        ret_char = <optimized out>
        typname = <optimized out>
#11 LWGEOM_GEOS_makeValid (gin=<optimized out>) at lwgeom_geos_clean.c:706
        gout = <optimized out>
        ret_char = <optimized out>
        typname = <optimized out>
#12 0x00007f436c9316a6 in lwgeom_make_valid (lwgeom_in=0x55bdae95b990) at lwgeom_geos_clean.c:898
        is3d = 0
        geosgeom = 0x55bdaf6a1eb0
        geosout = <optimized out>
        lwgeom_out = <optimized out>
        __PRETTY_FUNCTION__ = "lwgeom_make_valid"
#13 0x00007f436c881755 in ST_MakeValid (fcinfo=0x55bdae524520) at lwgeom_geos_clean.c:68
        in = 0x55bdaf282bc8
        out = <optimized out>
        lwgeom_in = <optimized out>
        lwgeom_out = <optimized out>
#14 0x000055bdadcba0b5 in ExecInterpExpr (state=0x55bdae524440, econtext=0x55bdae524980, isnull=<optimized out>) at ./build/../src/backend/executor/execExprInterp.c:649
        fcinfo = 0x55bdae524520
        args = 0x55bdae524540
        argno = <optimized out>
        d = <optimized out>
        op = <optimized out>
        resultslot = 0x0
        innerslot = <optimized out>
        outerslot = <optimized out>
        scanslot = 0x0
        dispatch_table = {0x55bdadcb90b4 <ExecInterpExpr+196>, 0x55bdadcb97f0 <ExecInterpExpr+2048>, 0x55bdadcb97c8 <ExecInterpExpr+2008>, 0x55bdadcb97a0 <ExecInterpExpr+1968>, 0x55bdadcb9768 <ExecInterpExpr+1912>, 0x55bdadcb9720 <ExecInterpExpr+1840>, 0x55bdadcb96f0 <ExecInterpExpr+1792>,
          0x55bdadcb96e0 <ExecInterpExpr+1776>, 0x55bdadcb9590 <ExecInterpExpr+1440>, 0x55bdadcb98b0 <ExecInterpExpr+2240>, 0x55bdadcb9890 <ExecInterpExpr+2208>, 0x55bdadcb9750 <ExecInterpExpr+1888>, 0x55bdadcb9560 <ExecInterpExpr+1392>, 0x55bdadcb9528 <ExecInterpExpr+1336>, 0x55bdadcb9820 <ExecInterpExpr+2096>,
          0x55bdadcb9850 <ExecInterpExpr+2144>, 0x55bdadcb96b8 <ExecInterpExpr+1736>, 0x55bdadcb9680 <ExecInterpExpr+1680>, 0x55bdadcb9638 <ExecInterpExpr+1608>, 0x55bdadcb9618 <ExecInterpExpr+1576>, 0x55bdadcb95f8 <ExecInterpExpr+1544>, 0x55bdadcb9040 <ExecInterpExpr+80>, 0x55bdadcb904b <ExecInterpExpr+91>,
          0x55bdadcb95b0 <ExecInterpExpr+1472>, 0x55bdadcb9070 <ExecInterpExpr+128>, 0x55bdadcb907b <ExecInterpExpr+139>, 0x55bdadcb9ad0 <ExecInterpExpr+2784>, 0x55bdadcb9ac0 <ExecInterpExpr+2768>, 0x55bdadcb9a90 <ExecInterpExpr+2720>, 0x55bdadcb9a88 <ExecInterpExpr+2712>, 0x55bdadcb9a70 <ExecInterpExpr+2688>,
          0x55bdadcb9a58 <ExecInterpExpr+2664>, 0x55bdadcb9a20 <ExecInterpExpr+2608>, 0x55bdadcb9a00 <ExecInterpExpr+2576>, 0x55bdadcb99d0 <ExecInterpExpr+2528>, 0x55bdadcb99c0 <ExecInterpExpr+2512>, 0x55bdadcb99a0 <ExecInterpExpr+2480>, 0x55bdadcb9948 <ExecInterpExpr+2392>, 0x55bdadcb9978 <ExecInterpExpr+2440>,
          0x55bdadcb9920 <ExecInterpExpr+2352>, 0x55bdadcb98f0 <ExecInterpExpr+2304>, 0x55bdadcb9fe0 <ExecInterpExpr+4080>, 0x55bdadcb98d0 <ExecInterpExpr+2272>, 0x55bdadcb9fc0 <ExecInterpExpr+4048>, 0x55bdadcba000 <ExecInterpExpr+4112>, 0x55bdadcb9f98 <ExecInterpExpr+4008>, 0x55bdadcb9eb0 <ExecInterpExpr+3776>,
          0x55bdadcb9e70 <ExecInterpExpr+3712>, 0x55bdadcb9e00 <ExecInterpExpr+3600>, 0x55bdadcb9f30 <ExecInterpExpr+3904>, 0x55bdadcb9e50 <ExecInterpExpr+3680>, 0x55bdadcba1f0 <ExecInterpExpr+4608>, 0x55bdadcb9de0 <ExecInterpExpr+3568>, 0x55bdadcb9dc8 <ExecInterpExpr+3544>, 0x55bdadcb9da0 <ExecInterpExpr+3504>,
          0x55bdadcb9d80 <ExecInterpExpr+3472>, 0x55bdadcb9cd0 <ExecInterpExpr+3296>, 0x55bdadcb9d30 <ExecInterpExpr+3392>, 0x55bdadcb9cb0 <ExecInterpExpr+3264>, 0x55bdadcb9c90 <ExecInterpExpr+3232>, 0x55bdadcb9c70 <ExecInterpExpr+3200>, 0x55bdadcb9c50 <ExecInterpExpr+3168>, 0x55bdadcb9c30 <ExecInterpExpr+3136>,
          0x55bdadcb9c10 <ExecInterpExpr+3104>, 0x55bdadcb9bf0 <ExecInterpExpr+3072>, 0x55bdadcb9d10 <ExecInterpExpr+3360>, 0x55bdadcb9f00 <ExecInterpExpr+3856>, 0x55bdadcb9b90 <ExecInterpExpr+2976>, 0x55bdadcb9150 <ExecInterpExpr+352>, 0x55bdadcb9bd0 <ExecInterpExpr+3040>, 0x55bdadcb9bb0 <ExecInterpExpr+3008>,
          0x55bdadcb9b70 <ExecInterpExpr+2944>, 0x55bdadcb9b48 <ExecInterpExpr+2904>, 0x55bdadcb9338 <ExecInterpExpr+840>, 0x55bdadcb9b10 <ExecInterpExpr+2848>, 0x55bdadcb9318 <ExecInterpExpr+808>, 0x55bdadcb9350 <ExecInterpExpr+864>, 0x55bdadcb92f0 <ExecInterpExpr+768>, 0x55bdadcb90e0 <ExecInterpExpr+240>,
          0x55bdadcb9260 <ExecInterpExpr+624>, 0x55bdadcb92b0 <ExecInterpExpr+704>, 0x55bdadcb9200 <ExecInterpExpr+528>, 0x55bdadcb91c0 <ExecInterpExpr+464>, 0x55bdadcb9478 <ExecInterpExpr+1160>, 0x55bdadcb9390 <ExecInterpExpr+928>, 0x55bdadcb9370 <ExecInterpExpr+896>, 0x55bdadcb91a0 <ExecInterpExpr+432>,
          0x55bdadcb90b0 <ExecInterpExpr+192>}

Attachments (1)

failed_makevalid.sql​ (3.4 MB ) - added by komzpa 4 years ago.

Change History (9)

by komzpa, 4 years ago

Attachment: failed_makevalid.sql​ added

comment:1 by komzpa, 4 years ago

GEOS 3.8 │ POSTGIS="3.0.0 r17983" [EXTENSION] PGSQL="110" GEOS="3.8.0-CAPI-1.13.1 " PROJ="6.2.1" LIBXML="2.9.4" LIBJSON="0.13.1" LIBPROTOBUF="1.3.2" WAGYU="0.4.3 (Internal)" │ 

#0  0x00007f9b3cfbe360 in geos::algorithm::PointLocator::computeLocation(geos::geom::Coordinate const&, geos::geom::Geometry const*) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
#1  0x00007f9b3cfbe597 in geos::algorithm::PointLocator::computeLocation(geos::geom::Coordinate const&, geos::geom::Geometry const*) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
#2  0x00007f9b3cfbe671 in geos::algorithm::PointLocator::locate(geos::geom::Coordinate const&, geos::geom::Geometry const*) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
#3  0x00007f9b3d043fa2 in geos::operation::overlay::OverlayOp::labelIncompleteNode(geos::geomgraph::Node*, int) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
#4  0x00007f9b3d044165 in geos::operation::overlay::OverlayOp::labelIncompleteNodes() () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
#5  0x00007f9b3d0453a2 in geos::operation::overlay::OverlayOp::computeOverlay(geos::operation::overlay::OverlayOp::OpCode) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
#6  0x00007f9b3d0456c9 in geos::operation::overlay::OverlayOp::getResultGeometry(geos::operation::overlay::OverlayOp::OpCode) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
#7  0x00007f9b3d045b4c in geos::operation::overlay::OverlayOp::overlayOp(geos::geom::Geometry const*, geos::geom::Geometry const*, geos::operation::overlay::OverlayOp::OpCode) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
#8  0x00007f9b3cfce0de in std::unique_ptr<geos::geom::Geometry, std::default_delete<geos::geom::Geometry> > geos::geom::BinaryOp<geos::operation::overlay::overlayOp>(geos::geom::Geometry const*, geos::geom::Geometry const*, geos::operation::overlay::overlayOp) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
#9  0x00007f9b3cfcbb23 in geos::geom::Geometry::difference(geos::geom::Geometry const*) const () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
#10 0x00007f9b3d05cec4 in geos::operation::valid::MakeValid::build(geos::geom::Geometry const*) () from /usr/lib/x86_64-linux-gnu/libgeos-3.8.0.so
#11 0x00007f9b3d3d7557 in GEOSMakeValid_r () from /usr/lib/x86_64-linux-gnu/libgeos_c.so.1
#12 0x00007f9b3d511c92 in ?? () from /usr/lib/postgresql/11/lib/postgis-3.so
#13 0x00007f9b3d4632ad in ST_MakeValid () from /usr/lib/postgresql/11/lib/postgis-3.so
#14 0x000055ea63adb3e7 in ?? ()
#15 0x000055ea63ba2604 in ?? ()
#16 0x000055ea63ba4ee9 in ?? ()
#17 0x000055ea63ba6097 in ?? ()
#18 0x000055ea63b36e5b in expression_tree_mutator ()
#19 0x000055ea63ba577a in ?? ()
#20 0x000055ea63b36fc5 in expression_tree_mutator ()
#21 0x000055ea63ba57bf in ?? ()
#22 0x000055ea63b36e5b in expression_tree_mutator ()
#23 0x000055ea63ba54ca in ?? ()
#24 0x000055ea63ba6097 in ?? ()
#25 0x000055ea63b36e5b in expression_tree_mutator ()
#26 0x000055ea63ba54ca in ?? ()
#27 0x000055ea63ba6097 in ?? ()
#28 0x000055ea63b36bde in expression_tree_mutator ()
#29 0x000055ea63ba577a in ?? ()
#30 0x000055ea63b36e5b in expression_tree_mutator ()
#31 0x000055ea63ba577a in ?? ()
#32 0x000055ea63ba6d2f in eval_const_expressions ()
#33 0x000055ea63b8ae78 in ?? ()
#34 0x000055ea63b92356 in subquery_planner ()
#35 0x000055ea63b939d9 in standard_planner ()
#36 0x000055ea63c59806 in pg_plan_query ()
#37 0x000055ea63c598f5 in pg_plan_queries ()
#38 0x000055ea63c59e0f in ?? ()
#39 0x000055ea63c5b625 in PostgresMain ()
#40 0x000055ea63bd4036 in ?? ()
#41 0x000055ea63bd4e89 in PostmasterMain ()
#42 0x000055ea63901140 in main ()

comment:2 by komzpa, 4 years ago

Summary: ST_MakeValid: endless uncancellable calculation → ST_MakeValid: long uncancellable calculation

comment:3 by komzpa, 4 years ago

Ran the case several times on GEOS 3.8. Run time varies from 2min40s to 6min40s. It ran hours on production server, might be because of resource starvation.

The thing is not cancellable.

comment:4 by Algunenano, 4 years ago

I understand this is a GEOS issue as it's not checking for interruptions for a long time, right? Is it reported there?

comment:5 by pramsey, 4 years ago

Milestone: PostGIS 2.5.4 → PostGIS 2.5.5

comment:6 by robe, 4 years ago

Milestone: PostGIS 2.5.5 → PostGIS 3.0.3

I asked on GEOS irc channel if this should be pegged as a GEOS issue.

comment:7 by pramsey, 4 years ago

I've added some interrupts to the GEOS MakeValid loop at ​geos:8a4f50ed.

comment:8 by pramsey, 4 years ago

Resolution: → fixed
Status: new → closed

Hearing no complaints (commit is here ​https://github.com/libgeos/geos/commit/8a4f50edddd9740742562b3f81ea2f5880eaf093) I am closing.

Note: See TracTickets for help on using tickets.