Rast_get_d_color() segfault

[hamish]

Hi,

I'm getting a segfault in grass7's r.what if I give it an out-of-region coordinate.

e.g.

#spearfish
r.what -n -r input=elevation.dem east_north=1,2

happens for both CELL, FCELL, and DCELL maps.

gdb:

(gdb) run -n -r input=elevation.dem east_north=1,2
easting|northing|site_name|elevation.dem|elevation.dem_color
** note ** 1 2 is outside your current window
[New Thread 0x7fedabfd06e0 (LWP 19658)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fedabfd06e0 (LWP 19658)]
0x00007fedab7836c6 in Rast_is_c_null_value (cellVal=0x184ef68) at null_val.c:211
211         return *cellVal == (CELL) 0x80000000;

(gdb) l
206        \return FALSE otherwise
207      */
208     int Rast_is_c_null_value(const CELL * cellVal)
209     {
210         /* Check if the CELL value matches the null pattern */
211         return *cellVal == (CELL) 0x80000000;
212     }
213     
214     /*!
215        \brief To check if a FCELL raster value is set to NULL

(gdb) bt
#0  0x00007f937bf6b6c6 in Rast_is_c_null_value (cellVal=0x127df68) at null_val.c:211
#1  0x00007f937bf6b67c in Rast_is_null_value (rast=0x127df68, data_type=0) at null_val.c:183
#2  0x00007f937bf722ac in Rast_get_d_value (rast=0x127df68, data_type=0) at raster.c:271
#3  0x00007f937bf5f984 in Rast__lookup_colors (raster=0x127df68, red=0x7fff847c559f "", grn=0x7fff847c559e "", 
    blu=0x7fff847c559d "\177", set=0x7fff847c559c "", n=0, colors=0x7fff847c6a10, mod=0, rules_only=0, data_type=0) at color_look.c:261
#4  0x00007f937bf5f55d in Rast_lookup_colors (raster=0x127df68, red=0x7fff847c559f "", grn=0x7fff847c559e "", blu=0x7fff847c559d "\177", 
    set=0x7fff847c559c "", n=1, colors=0x7fff847c6a10, map_type=0) at color_look.c:90
#5  0x00007f937bf5e546 in Rast_get_color (rast=0x127df68, red=0x7fff847c573c, grn=0x7fff847c5738, blu=0x7fff847c5734, 
    colors=0x7fff847c6a10, map_type=0) at color_get.c:44
#6  0x00007f937bf5e5b4 in Rast_get_c_color (rast=0x127df68, red=0x7fff847c573c, grn=0x7fff847c5738, blu=0x7fff847c5734, 
    colors=0x7fff847c6a10) at color_get.c:71
#7  0x0000000000402783 in main (argc=5, argv=0x7fff847d8c98) at main.c:384

DCELL is the same but different:

0x00007f630db80709 in Rast_is_d_null_value (dcellVal=0x22deb60) at null_val.c:258
258         return *dcellVal != *dcellVal;
(gdb) l
253        \return TRUE if DCELL raster value is NULL
254        \return FALSE otherwise
255      */
256     int Rast_is_d_null_value(const DCELL * dcellVal)
257     {
258         return *dcellVal != *dcellVal;
259     }
260     
261     /*!
262        \brief To insert null values into a map.
(gdb) bt
#0  0x00007f630db80709 in Rast_is_d_null_value (dcellVal=0x22deb60) at null_val.c:258
#1  0x00007f630db80698 in Rast_is_null_value (rast=0x22deb60, data_type=2) at null_val.c:189
#2  0x00007f630db872ac in Rast_get_d_value (rast=0x22deb60, data_type=2) at raster.c:271

thanks, Hamish

[hamish]

hmm, it happens in grass 6.4 and 6.5 as well.

apparently it has to do with the -r export colors flag.

Hamish

[hamish]

segfaults with x<= : r.what -nirf elevation.dem east=11070,1

works with x>= : r.what -nirf elevation.dem east=11071,1

[glynn]

Replying to hamish:

I'm getting a segfault in grass7's r.what if I give it an out-of-region coordinate.

Program received signal SIGSEGV, Segmentation fault.
...
(gdb) bt
...
#7  0x0000000000402783 in main (argc=5, argv=0x7fff847d8c98) at main.c:384

The code in question is:

383		    if (out_type[i] == CELL_TYPE)
384			Rast_get_c_color(&cell[i][cache[point].col],
385					     &red, &green, &blue, &ncolor[i]);
386		    else
387			Rast_get_d_color(&dcell[i][cache[point].col],
388					     &red, &green, &blue, &ncolor[i]);

> print cache[point].col
$3 = -19297

The above code needs to check that the indices are within the array's bounds.

[hamish]

as per comment:2, the issue was not just limited to trunk.

I'm not getting the segfault on my current system, but color_get.c hasn't had any fixes since the ticket was filed, so I assume it is still valid and would ask others to test.

thanks, Hamish