Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#4660 closed enhancement (fixed)

Sensible number output semantics

Reported by: Algunenano Owned by: Algunenano
Priority: medium Milestone: PostGIS 3.1.0
Component: postgis Version: master
Keywords: Cc:

Description (last modified by Algunenano)

In #4543 I enforced a limit on the number of digits output by lwprint_double. This limit was already in place but not always effective, which could lead to issues in other functions not calculating the necessary buffer correctly, so they would overestimate to be safe.

I set it to 15 digits, but it has come to my attention that some double numbers need up to 17 digits to be able to do the correct round trip, so I'm thinking on raising it from 15 to 17.

It should be available for both ST_AsText and ST_AsGeoJSON, but the user setting (or default) is still in place (so if they request 8 digits they should get 8 and no more).

Comes from 2 different mail threads:

Change History (8)

comment:1 by Algunenano, 2 years ago

Description: modified (diff)
Status: assignednew
Summary: Raise the max number of digits for double printing to 17Sensible number output semantics

comment:3 by Raúl Marín <git@…>, 2 years ago

Resolution: fixed
Status: newclosed

In 1343712/git:

Use the shortest representation when printing doubles

  • Use the shortest representation (enough to guarantee roundtrip).
  • Uses scientific notation for absolute numbers smaller than 1e-8. The previous behaviour was to output 0 for absolute values smaller than 1e-12 and fixed notation for anything bigger than that.
  • Uses scientific notation for absolute numbers greater than 1e+15 (same behaviour).
  • The precision parameter now also affects the scientific notation (before it was fixed [5-8]).
  • All output functions now respect the requested precision (without any limits).
  • The default precision is the same (9 for GeoJSON, 15 for everything else).

Many regress test changed mainly because of the fixes to the precision parameter, which is now respected as the amount of digits after the fixed point.

Closes https://github.com/postgis/postgis/pull/570
Closes #4660

comment:4 by Raúl Marín <git@…>, 2 years ago

In 9cac001/git:

Adapt regress tests to match Windows output

References #4660

comment:5 by Raúl Marín <git@…>, 2 years ago

In 6a0627c/git:

Adapt regress tests to match Windows output

References #4660

comment:6 by Raúl Marín <git@…>, 2 years ago

In 81b9af77/git:

Fix tests

Add missing update
References #4660

comment:7 by Raúl Marín <git@…>, 2 years ago

In 0d0cd341/git:

Attempt to fix output for Rasberry Pi

References #4660

comment:8 by Raúl Marín <git@…>, 2 years ago

In eb88a33/git:

Adapt output to work with Raspberry

References #4660

Note: See TracTickets for help on using tickets.