Opened 11 months ago

Closed 11 months ago

Last modified 9 months ago

#4600 closed defect (fixed)

ST_TileEnvelope default tile is imprecise

Reported by: Algunenano Owned by: Algunenano
Priority: medium Milestone: PostGIS 3.0.1
Component: postgis Version: 3.0.x
Keywords: Cc:


When checking the initial tile I get integer precision, which is odd to say the least:

select ST_AsText(ST_TileEnvelope(0, 0, 0));
 POLYGON((-20037510 -20037510,-20037510 20037510,20037510 20037510,20037510 -20037510,-20037510 -20037510))
(1 row)

Carto's function seems to do this ok:

Select ST_AsText(cdb_xyz_extent(0, 0, 0));
 POLYGON((-20037508.3427892 20037508.3427892,-20037508.3427892 -20037508.3427892,20037508.3427892 -20037508.3427892,20037508.3427892 20037508.3427892,-20037508.3427892 20037508.3427892))

This seems related to the limited precision of floats, which is how bbox are stored.

The problem is that this imprecision is found in the base tile, so its error is translated to all other tiles and becomes evident in any tile around the borders.

The difference is low for 0/0/0:

Select ST_Area(ST_Intersection(ST_TileEnvelope(0, 0, 0), cdb_xyz_extent(0, 0, 0))) / ST_Area(cdb_xyz_extent(0, 0, 0));
(1 row)

But it's not for certain lower tiles:

Select ST_Area(ST_Intersection(ST_TileEnvelope(18, 262143, 0), cdb_xyz_extent(262143, 0, 18))) / ST_Area(cdb_xyz_extent(262143, 0, 18));
(1 row)

That's over 2% error.

Although it isn't great, I suggest recalculating the bbox of the input geometry the precision of a 64b floating point (vs 32b). With that, the same tile (18/0/0) is 0.999999996832113 equivalent, which is what we would expect since CARTO's function uses a slightly different geometry and algorithm.

Change History (5)

comment:1 Changed 11 months ago by Algunenano

PR to always calculate the bbox in

comment:2 Changed 11 months ago by Raúl Marín <git@…>

In 939c306/git:

ST_TileEnvelope: Improve precision by calculating the input bbox

References #4600
References #4602


comment:3 Changed 11 months ago by Raúl Marín <git@…>

In 81754c5/git:

LWGEOM_addpoint: Accept -1 as a valid position

References #4600

comment:4 Changed 11 months ago by Raúl Marín <git@…>

Resolution: fixed
Status: assignedclosed

In 531bc441/git:

ST_TileEnvelope: Improve precision by calculating the input bbox

Closes #4600

comment:5 Changed 9 months ago by Raúl Marín <git@…>

In b73f673/git:

Reduce output precision in ST_TileEnvelope test

The output is slightly different in 32bit plaftorms

References #4619
References #4600

Note: See TracTickets for help on using tickets.