Opened 5 years ago

Closed 4 years ago

Last modified 4 years ago

#3713 closed defect (fixed)

Data loss on big encoded polyline

Reported by: ismail BASKIN Owned by: dbaston
Priority: medium Milestone: PostGIS 2.3.5
Component: postgis Version: 2.3.x
Keywords: Cc: dbaston

Description

it is true as expected

--it is true as expected
SELECT ST_IsClosed(ST_LineFromEncodedPolyline('guhiFwmpqEs]fJwmAbmAcmAngAwGve@wGrcBg|@~bBo_@bQ{c@fTos@vLs]bGscBsD{uAkf@cuB{aAkMsD{kAc[os@rDs`AzOs{@zTwj@zOkCnZoPntBoPf|@oKzOseAj}A{c@zw@c[bj@s]bmAcQvj@oAfEcVfw@s{@vvDce@~gBwQfr@_b@rv@s]ve@s{@nU{}B~C{c@kCsl@gEkCSo|CsDkHoAk_A_SklBw`@s`AgJchA?kH?_eAoUon@{Os]sIkiASs`Ab[cpBf|@_`Ajf@wBz@kMvGgYvrAbBz^rDn}@jf@znBbLrhBjCjwDg@bBsD~McBnF{Jf^wVr{@{EfO{zAbuBcG~H{zA~bB{Of|@_b@jxAos@f|@od@jH_q@bLowCnZkbB?kk@Ss`Ab[s`A~bBwhA~oCco@f{CgOb`@kMb[{JvL{JvLkRnUcrAzOwmAgJ_`AzT_b@zaAwj@~p@gYfw@wGbrAbBrq@rDrhBvGf|@nAjnArNf^fw@zEjf@nF~f@zErv@v~@jz@rgEvV~eCfOrkCvLrIsDj`HrNzY~p@bwAjRb`@zJ{@jW_DvrAgOfEoAbcAgY~a@g^jCcGjWco@nPw[zOoZrXg^jH{JfYs]zJkMbo@sb@rcBkbBnZ{YfYgObGSzm@oA~oCbhAj_An_@faF~iArI~Cve@bQrrBbt@bQnUjRvVbLzO~sArtAnx@nU~a@be@z^be@ni@fh@rXfc@jWvVjRvo@zTfw@~Wnx@fc@v[be@bVvVzJb~@j\jp@R~Hzc@rIja@z@rIjCrSrDbj@rD~f@zE~a@vBb[vBj\bBz^nAnZSz^?~\Srb@wBnn@oArXcBni@kCf^?fc@SnFcBb`@g@~f@_Dbe@_Drg@kCn_@{Ej\_Drb@kCfYwBfJz^fmEr]~sFzTrgE?z_G??jC~oCbLnUvj@rDbrArDraCfJjpEsD~bBjH??zYz@zxBfEbpBzOjlBnUr~Ave@jlBvrAjiAz|@nvAjsAzr@nlAf|@~p@vj@fJ~_Ab[zh@r]z^vVjHzEzxBjxAznBbcAju@n_@f}BngA~gBnUve@jf@vmAngAbQnZ~\~k@r~AfJzw@sDvmAkk@nn@kk@fYcmAjCkxAkMseAwBcQcGkf@oPscBzT{fAbL_l@vj@gOzY{@r{@wB~_AgJbj@?~C?nqA_l@jH_DjbBogA~iAnUr{@fJ~nAr]nFbBvmAsD~M_DnbAoUjf@_l@zw@_q@vhA{T~dAgJzzAfJr~A~k@~dA~p@vmArq@fEz@vy@~Rbo@_Dbo@w`@?kk@oPw`@co@{Ts`A{OcmAkf@{EkH_q@wcAwGkxAbL{sBvj@gnBvhAkxAzE{@zc@kHns@g|@jk@kRbGoAbcAgfAf@g@ns@ogAfYogA?cmAwGclDcLwdCwGgnBoPcmA?_~AkHg^oKgh@_b@_~As]sIgYf@gxBv`@kdAbt@oyBbwAgc@Rw[{JoZgY_`AkeCos@okDk_AkkEkM{m@cBcGkp@gbCs]suC?scBbLcrAbo@_~Azw@shBvQkf@zh@stAjCwGjCcmAkf@czBgw@_cB_eAczB_eAgnBwL_l@gT{aA{TwdCkCwdC?_bERsDnK_~Ar]ogAns@g|@zEwBjsAsg@jlB_q@rg@_g@nZs]nU_X~\seArIgh@kHgwEwGkk@_b@{aAcLsq@{w@kxAcLogAka@g|@{^cj@w`@gm@{^os@k\co@oPg|@wBwLcQwy@wGwrARgEvBo~BkCwrA{TgnB_Ik\sS_{@?we@oPsq@sXc`@os@wrAg_Bwy@od@oUsg@_X{zAsv@crA{|@gw@scB_l@ovA{JwV{aAsrBkCwGgfAczBwt@c|AS_NbGos@g@cVwG_b@oP_SkRgTg_Bc[s~A_l@s`AgJwj@rDco@rv@g^j_AsSfh@g|@rq@_eAS{ToZ{TwrAoKkf@s]cmA{JcQcVod@wpB?gOkHkk@kWgYkk@oPsq@'));
 

it is false, because ST_AsEncodedPolyline-ST_LineFromEncodedPolyline corrupt data.

--it is false, because ST_AsEncodedPolyline-ST_LineFromEncodedPolyline corrupt data.
SELECT ST_IsClosed(ST_LineFromEncodedPolyline(ST_AsEncodedPolyline(ST_LineFromEncodedPolyline('guhiFwmpqEs]fJwmAbmAcmAngAwGve@wGrcBg|@~bBo_@bQ{c@fTos@vLs]bGscBsD{uAkf@cuB{aAkMsD{kAc[os@rDs`AzOs{@zTwj@zOkCnZoPntBoPf|@oKzOseAj}A{c@zw@c[bj@s]bmAcQvj@oAfEcVfw@s{@vvDce@~gBwQfr@_b@rv@s]ve@s{@nU{}B~C{c@kCsl@gEkCSo|CsDkHoAk_A_SklBw`@s`AgJchA?kH?_eAoUon@{Os]sIkiASs`Ab[cpBf|@_`Ajf@wBz@kMvGgYvrAbBz^rDn}@jf@znBbLrhBjCjwDg@bBsD~McBnF{Jf^wVr{@{EfO{zAbuBcG~H{zA~bB{Of|@_b@jxAos@f|@od@jH_q@bLowCnZkbB?kk@Ss`Ab[s`A~bBwhA~oCco@f{CgOb`@kMb[{JvL{JvLkRnUcrAzOwmAgJ_`AzT_b@zaAwj@~p@gYfw@wGbrAbBrq@rDrhBvGf|@nAjnArNf^fw@zEjf@nF~f@zErv@v~@jz@rgEvV~eCfOrkCvLrIsDj`HrNzY~p@bwAjRb`@zJ{@jW_DvrAgOfEoAbcAgY~a@g^jCcGjWco@nPw[zOoZrXg^jH{JfYs]zJkMbo@sb@rcBkbBnZ{YfYgObGSzm@oA~oCbhAj_An_@faF~iArI~Cve@bQrrBbt@bQnUjRvVbLzO~sArtAnx@nU~a@be@z^be@ni@fh@rXfc@jWvVjRvo@zTfw@~Wnx@fc@v[be@bVvVzJb~@j\jp@R~Hzc@rIja@z@rIjCrSrDbj@rD~f@zE~a@vBb[vBj\bBz^nAnZSz^?~\Srb@wBnn@oArXcBni@kCf^?fc@SnFcBb`@g@~f@_Dbe@_Drg@kCn_@{Ej\_Drb@kCfYwBfJz^fmEr]~sFzTrgE?z_G??jC~oCbLnUvj@rDbrArDraCfJjpEsD~bBjH??zYz@zxBfEbpBzOjlBnUr~Ave@jlBvrAjiAz|@nvAjsAzr@nlAf|@~p@vj@fJ~_Ab[zh@r]z^vVjHzEzxBjxAznBbcAju@n_@f}BngA~gBnUve@jf@vmAngAbQnZ~\~k@r~AfJzw@sDvmAkk@nn@kk@fYcmAjCkxAkMseAwBcQcGkf@oPscBzT{fAbL_l@vj@gOzY{@r{@wB~_AgJbj@?~C?nqA_l@jH_DjbBogA~iAnUr{@fJ~nAr]nFbBvmAsD~M_DnbAoUjf@_l@zw@_q@vhA{T~dAgJzzAfJr~A~k@~dA~p@vmArq@fEz@vy@~Rbo@_Dbo@w`@?kk@oPw`@co@{Ts`A{OcmAkf@{EkH_q@wcAwGkxAbL{sBvj@gnBvhAkxAzE{@zc@kHns@g|@jk@kRbGoAbcAgfAf@g@ns@ogAfYogA?cmAwGclDcLwdCwGgnBoPcmA?_~AkHg^oKgh@_b@_~As]sIgYf@gxBv`@kdAbt@oyBbwAgc@Rw[{JoZgY_`AkeCos@okDk_AkkEkM{m@cBcGkp@gbCs]suC?scBbLcrAbo@_~Azw@shBvQkf@zh@stAjCwGjCcmAkf@czBgw@_cB_eAczB_eAgnBwL_l@gT{aA{TwdCkCwdC?_bERsDnK_~Ar]ogAns@g|@zEwBjsAsg@jlB_q@rg@_g@nZs]nU_X~\seArIgh@kHgwEwGkk@_b@{aAcLsq@{w@kxAcLogAka@g|@{^cj@w`@gm@{^os@k\co@oPg|@wBwLcQwy@wGwrARgEvBo~BkCwrA{TgnB_Ik\sS_{@?we@oPsq@sXc`@os@wrAg_Bwy@od@oUsg@_X{zAsv@crA{|@gw@scB_l@ovA{JwV{aAsrBkCwGgfAczBwt@c|AS_NbGos@g@cVwG_b@oP_SkRgTg_Bc[s~A_l@s`AgJwj@rDco@rv@g^j_AsSfh@g|@rq@_eAS{ToZ{TwrAoKkf@s]cmA{JcQcVod@wpB?gOkHkk@kWgYkk@oPsq@'))));

Change History (13)

comment:1 Changed 5 years ago by woodbri

The encode and decode function take a precision=5 argument. This means that it truncates the floating point numbers at that number of significant decimal point digits. So if you encode and decode a linestring you will not get back "exactly" the same linestring. This is the definition of the encoding and decoding functions. So there is no error here other then understanding what the function is doing.

comment:2 Changed 5 years ago by ismail BASKIN

Example polyline encoded on 5 precision (34.4958 38.3882,34.494 38.3931...). So these functions should not truncate values.

Version 0, edited 5 years ago by ismail BASKIN (next)

comment:3 Changed 5 years ago by woodbri

Thanks for example, this will help someone narrow down the issue.

comment:4 Changed 4 years ago by ismail BASKIN

Is there any improvement?

comment:5 Changed 4 years ago by robe

Cc: dbaston added

No. Still get same answer in PostGIS 2.4 dev. @dbaston any thoughts on this. I vaguely remember a limitation of representing things like polygons so wondering if that affects closed rings as well though why LineFrom? wouldn't suffer the same issue doesn't make sense.

comment:6 Changed 4 years ago by robe

Owner: changed from pramsey to dbaston

comment:7 Changed 4 years ago by robe

Milestone: PostGIS 2.3.3PostGIS 2.3.4

I'm guessing this won't be done anytime soon. Push back if you can get to it in next day or 2.

comment:8 Changed 4 years ago by pramsey

Milestone: PostGIS 2.3.4PostGIS 2.3.5

comment:9 Changed 4 years ago by pramsey

Here's an example that is small enough to debug:

with d as (select 'SRID=4326;LINESTRING(33.6729 38.7071,33.6692 38.701,33.6673 38.6972,33.6626 38.6871)'::geometry as g),
e as (
   select g, ST_LineFromEncodedPolyline(ST_AsEncodedPolyline(g)) as gl
   from d
   ) 
select 
  st_astext(g), ST_AsEncodedPolyline(g), st_npoints(g), 
  st_astext(gl), st_npoints(gl)
from e

yuck.

comment:10 Changed 4 years ago by pramsey

In 16130:

Support google line encodings that include a backslash character (References #3713)

comment:11 Changed 4 years ago by pramsey

In 16131:

Support google line encodings that include a backslash character (References #3713)

comment:12 Changed 4 years ago by pramsey

In 16132:

Support google line encodings that include a backslash character (References #3713)

comment:13 Changed 4 years ago by pramsey

Resolution: fixed
Status: newclosed

There were actually two things going on. The original author thought that it would be smart to escape the backslash character in the output, but actually the format didn't like that, it preferred the raw character. Also the postgis/ side caller was prematurely freeing the database memory underneath, which caused failures *sometimes*. Fix applied to trunk, 2.4, 2.3.

Last edited 4 years ago by pramsey (previous) (diff)
Note: See TracTickets for help on using tickets.