Opened 14 years ago

Last modified 8 years ago

#771 new task

use longer ints for counting raster cells

Reported by: hamish Owned by: grass-dev@…
Priority: normal Milestone: 6.4.6
Component: Raster Version: svn-develbranch6
Keywords: overflow Cc:
CPU: Unspecified Platform: All

Description

Hi,

when region sizes exceed a bit more than 45000x45000 plain int no longer has the bitspace to keep track of the number of cells and overflows. 'unsigned int' only buys us a little time until region size exceeds 65536x65536.

in these cases 'unsigned long long' could be used to make the size bigger. not all compilers know about long long though so ./configure does a check for it and sets a macro.

  • r.stats and r.univar are a priority here.

r.info and g.region seem to have working implimentations,

example code from 6.5's g.region:

#ifdef HAVE_LONG_LONG_INT
            fprintf(stdout, "%-*s %lld\n", width, "cells:",
                    (long long)window->rows * window->cols);
            if (print_flag & PRINT_3D)
                fprintf(stdout, "%-*s %lld\n", width, "3dcells:",
                        (long long)window->rows3 * window->cols3 *
                        window->depths);
#else
            fprintf(stdout, "%-*s %ld\n", width, "cells:",
                    (long)window->rows * window->cols);
            if (print_flag & PRINT_3D)
                fprintf(stdout, "%-*s %ld\n", width, "3dcells:",
                        (long)window->rows3 * window->cols3 * window->depths);
#endif

r.info:

#ifdef HAVE_LONG_LONG_INT
            compose_line(out, "  Total Cells:  %llu",
                         (unsigned long long)cellhd.rows * cellhd.cols);
#else
            compose_line(out,
                         "  Total Cells:  %lu (accuracy - see r.info manual)",
                         (unsigned long)cellhd.rows * cellhd.cols);
#endif

r.info man page:

Some standards (ISO-C90) and compilers do not support the 'long long' type 
as a 64-bit type. In the case that GRASS was built with such a compiler, 
an accuracy message may be displayed in the output of <em>r.info</em> 
after Total Cells:

we should standardize on something. any reason not to make it unsigned?

comments?

Hamish

Change History (2)

in reply to:  description comment:1 by glynn, 14 years ago

Replying to hamish:

r.info man page:

Some standards (ISO-C90) and compilers do not support the 'long long' type 
as a 64-bit type.

Just to clarify: C89 doesn't support it at all; the compiler will report a syntax error if the code contains "long long".

we should standardize on something. any reason not to make it unsigned?

For counts, using an unsigned type makes sense. Using "unsigned long" will at least give you 64-bit values on 64-bit systems.

One caveat is that comparing signed and unsigned values will cast the signed value to an unsigned value, which results in undefined behaviour if the signed value is negative. So you need to use e.g.:

	if (sval < 0 || (unsigned) sval < uval) ...
	if (sval < 0 || (unsigned) sval <= uval) ...
	if (sval >= 0 && (unsigned) sval == uval) ...
	if (sval >= 0 && (unsigned) sval >= uval) ...
	if (sval >= 0 && (unsigned) sval > uval) ...

to ensure that the signed-unsigned comparison is only performed when the signed value is non-negative.

Another caveat is the difference between two arbitrary unsigned values cannot be represented by the equivalent signed type.

comment:2 by neteler, 8 years ago

Milestone: 6.4.06.4.6
Note: See TracTickets for help on using tickets.