PostGIS 3.1 + Postgres 13 MVT crash

[komzpa]

[22:32] <Komzpa> Huston we have a problem
[22:32] <Komzpa> https://gist.github.com/Komzpa/c5f1b970d45cd3aa11e4efc60c38295d
[22:34] <Komzpa> that might be because of mix of a debian postgres + ubuntu postgis, let me revert both to ubuntu pgdg and recheck
[22:34] <strk> doesn't tell much, did you look at that line in postgresql sourcecode ?
[22:37] <Komzpa> https://github.com/postgres/postgres/blob/master/src/backend/executor/execExpr.c#L3214
[22:37] <Komzpa> Assert(as->d.agg_strict_input_check.jumpnull == -1);
[22:41] <Komzpa> the only aggregate in the query is ST_AsMVT(q, 'stats', 8192, 'geom')
[22:46] <RhodiumToad> that's ... not good
[22:47] <Komzpa> 22:46:02 [gis] > select ST_AsMVT(q, 'stats', 8192, 'geom') from (select 1 as id, 'POINT (0 0)' as geom) q;
[22:47] <Komzpa> ERROR:  XX000: Unknown geometry type: 0 - Unknown
[22:47] <Komzpa> LOCATION:  pg_error, lwgeom_pg.c:250
[22:47] <Komzpa> Time: 0,610 ms
[22:47] <Komzpa> 22:46:15 [gis] > select ST_AsMVT(q, 'stats', 8192, 'geom') from (select 1 as id, 'POLYGON(1 1, 3 3, 4 0, 1 1)' as geom) q;
[22:47] <Komzpa> ERROR:  XX000: Unknown geometry type: 857748256 - Invalid type
[22:47] <Komzpa> LOCATION:  pg_error, lwgeom_pg.c:250
[22:47] <Komzpa> Time: 0,634 ms
[22:47] <Komzpa> does not look expected to me
[22:48] <Komzpa> no, going back and syncing versions does not help
[22:48] <RhodiumToad> I don't think that's relevant to your failed assertion
[22:49] <Komzpa> the spot is the same
[22:49] <RhodiumToad> this looks like an oversight in the code?
[22:49] <Komzpa> I mean, it's a minimized example of my query that's already misbehaving
[22:49] <Komzpa> but not yet to a crash
[22:49] <RhodiumToad> right, but if my guess is right, the crash depends on amount of data
[22:50] <RhodiumToad> since it's related to aggregate serialize/deserialize functions, which are used in parallel aggregation
[22:51] <RhodiumToad> you're testing on 13.1?
[22:53] <Komzpa> yes
[22:57] <Komzpa> RhodiumToad: does this sound like postgis or postgres issue?
[22:57] <RhodiumToad> Komzpa: does that aggregate function have a deserialize function that is not declared as strict?
[22:58] <RhodiumToad> my initial suspicion (not confirmed) is that it's a postgres issue, but triggered by the presence of a non-strict deserialize function (all the builtin deserialize functions are strict)
[22:58] <Komzpa> yes, since PostGIS 2.5.0 https://github.com/postgis/postgis/blob/8b13c3e2f8366d902dbf516ec17de09ae84361f4/postgis/postgis.sql.in#L4836
[22:59] <Komzpa> it was working okayish before postgis 3.1 upgrafe
[23:02] <Komzpa> 23:02:45 [gis] > alter function pgis_asmvt_deserialfn strict;
[23:03] <Komzpa> RhodiumToad: as I understand that is expected to help?
[23:06] <RhodiumToad> well it might break the handling of nulls, depending on how the other funcs for the aggregate are written
[23:07] <RhodiumToad> ... did it make any difference?
[23:08] <Komzpa> tonight there was crash on second 2, now job runs for 5 minutes finely already
[23:08] <Komzpa> so at least it helped the crash

[darkblueb]

gist contents

POSTGIS="3.1.0 5e2af69" [EXTENSION] PGSQL="130" GEOS="3.9.0-CAPI-1.14.0" SFCGAL="1.3.7" PROJ="7.1.0" GDAL="GDAL 3.1.3, released 2020/09/01" LIBXML="2.9.10" LIBJSON="0.15" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)" TOPOLOGY RASTER │
 PostgreSQL 13.1 (Debian 13.1-1.pgdg+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.0-16) 10.2.0, 64-bit 




TRAP: FailedAssertion("false", File: "/build/postgresql-13-Ki8OqG/postgresql-13-13.1/build/../src/backend/executor/execExpr.c", Line: 3214)
postgres: 13/main: gis gis [local] COPY(ExceptionalCondition+0x7d)[0x559f9d9660dd]
postgres: 13/main: gis gis [local] COPY(ExecBuildAggTrans+0x8c3)[0x559f9d66f153]
postgres: 13/main: gis gis [local] COPY(ExecInitAgg+0x1825)[0x559f9d691f55]
postgres: 13/main: gis gis [local] COPY(ExecInitNode+0x199)[0x559f9d681749]
postgres: 13/main: gis gis [local] COPY(standard_ExecutorStart+0x35b)[0x559f9d67a61b]
postgres: 13/main: gis gis [local] COPY(+0x242dc2)[0x559f9d5ecdc2]
postgres: 13/main: gis gis [local] COPY(+0x2431f5)[0x559f9d5ed1f5]
postgres: 13/main: gis gis [local] COPY(DoCopy+0xd3)[0x559f9d5f0d13]
postgres: 13/main: gis gis [local] COPY(standard_ProcessUtility+0x717)[0x559f9d829ba7]
postgres: 13/main: gis gis [local] COPY(+0x47c6e4)[0x559f9d8266e4]
postgres: 13/main: gis gis [local] COPY(+0x47d1e2)[0x559f9d8271e2]
postgres: 13/main: gis gis [local] COPY(PortalRun+0x1c2)[0x559f9d828222]
postgres: 13/main: gis gis [local] COPY(+0x4796a0)[0x559f9d8236a0]
postgres: 13/main: gis gis [local] COPY(PostgresMain+0x1a53)[0x559f9d825513]
postgres: 13/main: gis gis [local] COPY(+0x3e5966)[0x559f9d78f966]
postgres: 13/main: gis gis [local] COPY(PostmasterMain+0xd78)[0x559f9d790858]
postgres: 13/main: gis gis [local] COPY(main+0x796)[0x559f9d46c326]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf2)[0x7fe0d30e9cb2]
postgres: 13/main: gis gis [local] COPY(_start+0x2a)[0x559f9d46c3ca]

[strk]

This should be fixed in current 3.1 and master branches. Commit references left in #4814

[komzpa]

Hello strk please post commit links here as I read through the history on master branch and found nothing that seems to address this.

[strk]

Sorry I mistaken this as being the MakeValid crash. Did you not file a ticket about that ? Just reopened the one about NaN handling ?

[strk]

The commit I was referring to: [6d74ddcf9dd40c8dee9dfacbb4da31a7e9eead89/git]

[Algunenano]

I understand this is fixed with the MakeValid / empty geos geometries thingy. If not it needs a repro case.

[komzpa]

No, it is not a MakeValid issue and why is it closed as it two times in a row as if it is what am I doing wrong why is nobody reading I am in pain because of it sorry.

This one is postgres issue: ​https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=a3367aa3c4552170004c92369681730d85e384c6

Workaround for whoever bumps into this: alter function pgis_asmvt_deserialfn strict;

[Algunenano]

No, it is not a MakeValid? issue and why is it closed as it two times in a row as if it is what am I doing wrong why is nobody reading I am in pain because of it sorry.

Sorry, it wasn't clear to me that this was a Postgresql issue that they were handling.

In that case it should be marked as a Postgresql issue and point to their mailing list and and not assigned to me.