Changes between Initial Version and Version 1 of Ticket #5653


Ignore:
Timestamp:
Sep 17, 2014, 4:27:51 PM (10 years ago)
Author:
Lucian Plesea
Comment:

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #5653 – Description

    initial v1  
    33This is due to the following two lines in gcore/overview.cpp, in GDALRegenerateOverviewsMultiBand(), lines 1993 and 1994 currently:
    44
    5 ''int nFullResXChunk = (nDstBlockXSize * nSrcWidth) / nDstWidth;
    6 int nFullResYChunk = (nDstBlockYSize * nSrcHeight) / nDstHeight;
     5''int nFullResXChunk = (nDstBlockXSize * nSrcWidth) / nDstWidth;''
     6
     7''int nFullResYChunk = (nDstBlockYSize * nSrcHeight) / nDstHeight;
    78''
    89
    9 Most of the time the result is correct, nSrcWidth/nDstWidth is commonly 2/1.  When nSrcWidth is odd, nDstWidth is rounded up, thus the ratio is a bit under 2.  Since all variables are integers, the end result is rounded down, the result ends up being off by one.  For example (512*1023)/512 results in 1023 instead of the correct 1024.  The brackets makes this error less likely.  It escaped detection because it is relatively infrequent and the effect is noticeable only then the sizes are large, the stepping error having time to accumulate.
     10Most of the time the result is correct, nSrcWidth/nDstWidth is commonly 2/1.  When nSrcWidth is odd, nDstWidth is rounded up, thus the ratio is a bit under 2.  Since all variables are integers, the end result is rounded down, the result ends up being off by one.  For example (512*1023)/512 results in 1023 instead of the correct 1024.  The brackets makes this error less likely.  It escaped detection because it is relatively infrequent and the effect is noticeable only when the sizes are large, the stepping error having time to accumulate.
    1011
    1112The proposed fix is to use floating point and round up to an integer, which also takes care of the potential integer overflow in the original formula.  This calculation is not in an inner loop, so efficiency is not an issue.
    1213
    13 ''int nFullResXChunk = int( double(nDstBlockXSize) * nSrcWidth / nDstWidth + 0.5);
    14 int nFullResYChunk = int( double(nDstBlockYSize) * nSrcHeight / nDstHeight + 0.5;''
     14''int nFullResXChunk = int( double(nDstBlockXSize) * nSrcWidth / nDstWidth + 0.5);''
    1515
     16''int nFullResYChunk = int( double(nDstBlockYSize) * nSrcHeight / nDstHeight + 0.5;''
     17