Changes between Version 1 and Version 2 of Ticket #1127, comment 17


Ignore:
Timestamp:
Oct 18, 2021, 4:05:07 PM (3 years ago)
Author:
mdavis

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #1127, comment 17

    v1 v2  
    33As mentioned before, the precision reduction code is tailored towards situations where the actual numerical precision of the input numbers is reduced.  Also, the gridsize argument is actually inverted for use by the actual code - so that 612 becomes 0.001633986928105.  This is not reducing the precision of the input numbers, only changing them. 
    44
    5 Instead, try using a slightly larger grid size which is a simple multiple of 5, 10, or 2.  For instance, try using 800 instead of 612, or 1250 instead of 1222.99245256
     5Instead, try using a slightly larger grid size which is a "simple" multiple of 5, 10, and 2.  I have confirmed this works for all 3 cases provided so far on this ticket.
     6
     7* 612 -> 800
     8* 152.87406 -> 200
     9* 1222.99245256 -> 1250