Opened 5 years ago

Closed 5 years ago

Last modified 4 years ago

#4600 closed defect (fixed)

ST_TileEnvelope default tile is imprecise

Reported by: Algunenano Owned by: Algunenano
Priority: medium Milestone: PostGIS 3.0.1
Component: postgis Version: 3.0.x
Keywords: Cc:


When checking the initial tile I get integer precision, which is odd to say the least:

select ST_AsText(ST_TileEnvelope(0, 0, 0));
 POLYGON((-20037510 -20037510,-20037510 20037510,20037510 20037510,20037510 -20037510,-20037510 -20037510))
(1 row)

Carto's function seems to do this ok:

Select ST_AsText(cdb_xyz_extent(0, 0, 0));
 POLYGON((-20037508.3427892 20037508.3427892,-20037508.3427892 -20037508.3427892,20037508.3427892 -20037508.3427892,20037508.3427892 20037508.3427892,-20037508.3427892 20037508.3427892))

This seems related to the limited precision of floats, which is how bbox are stored.

The problem is that this imprecision is found in the base tile, so its error is translated to all other tiles and becomes evident in any tile around the borders.

The difference is low for 0/0/0:

Select ST_Area(ST_Intersection(ST_TileEnvelope(0, 0, 0), cdb_xyz_extent(0, 0, 0))) / ST_Area(cdb_xyz_extent(0, 0, 0));
(1 row)

But it's not for certain lower tiles:

Select ST_Area(ST_Intersection(ST_TileEnvelope(18, 262143, 0), cdb_xyz_extent(262143, 0, 18))) / ST_Area(cdb_xyz_extent(262143, 0, 18));
(1 row)

That's over 2% error.

Although it isn't great, I suggest recalculating the bbox of the input geometry the precision of a 64b floating point (vs 32b). With that, the same tile (18/0/0) is 0.999999996832113 equivalent, which is what we would expect since CARTO's function uses a slightly different geometry and algorithm.

Change History (5)

comment:1 by Algunenano, 5 years ago

PR to always calculate the bbox in

comment:2 by Raúl Marín <git@…>, 5 years ago

In 939c306/git:

ST_TileEnvelope: Improve precision by calculating the input bbox

References #4600
References #4602


comment:3 by Raúl Marín <git@…>, 5 years ago

In 81754c5/git:

LWGEOM_addpoint: Accept -1 as a valid position

References #4600

comment:4 by Raúl Marín <git@…>, 5 years ago

Resolution: fixed
Status: assignedclosed

In 531bc441/git:

ST_TileEnvelope: Improve precision by calculating the input bbox

Closes #4600

comment:5 by Raúl Marín <git@…>, 4 years ago

In b73f673/git:

Reduce output precision in ST_TileEnvelope test

The output is slightly different in 32bit plaftorms

References #4619
References #4600

Note: See TracTickets for help on using tickets.