[raster] ST_PixelAsPolygons does not obey exclude_nodata_value=TRUE

[alfred]

ST_PixelAsPolygons and all related functions (Ex: ST_PixelAsCentroids) always iterate over all pixels, even the ones with nodatavalue. Setting the "exclude_nodata_value" parameter does not seem to affect the result at all.

Seen with PostGIS 2.1.0 final. Patch for this will be greatly appreciated.

[dustymugs]

When exclude_nodata_value = true, the value for NODATA pixels are returned as NULL. When exclude_nodata_value = false, the value for NODATA pixels are returned as the number designated the NODATA value.

You can get rid of those NULL values by adding a where clause…

SELECT (gv).x, (gv).y, (gv).val, ST_AsText((gv).geom) geom
FROM (SELECT ST_PixelAsPolygons(
                 ST_SetValue(ST_SetValue(ST_AddBand(ST_MakeEmptyRaster(2, 2, 0, 0, 0.001, -0.001, 0.001, 0.001, 4269), 
                                                    '8BUI'::text, 1, 0), 
                                         2, 2, 10), 
                             1, 1, NULL), 1, true
) gv 
) foo
WHERE (gv).val IS NOT NULL

[dustymugs]

Looking at how the behavior changed between 2.0 and 2.1 (the exclude_nodata_value parameter was added in 2.1), ST_PixelAsPolygons() has always returned NULL value for a pixel set to the NODATA value. This behavior is present before 2.0.

The exclude_nodata_value parameter was added in 2.1 so that ALL pixel values would be returned regardless of a pixel being NODATA or not.

If people think that the behavior should be changed so that when exclude_nodata_value = true NODATA pixels are not returned, that change can be made for 2.2.

[pracine]

There are three options here:

1-We don't want NODATA pixels to be returned as polygons

2-We want them to be returned but set to NULL

3-We want them to be returned but set as the NODATA value (e.g. 0), not NULL

Option 3 should be possible by deactivating the nodatavalue of the raster with ST_SetBandNoDataValue(rast, null).

Option 1 and 2 should be controled by exclude_nodata_value and this is the only thing that exclude_nodata_value should control.

Is this the current behavior?

[dustymugs]

In 2.0, only option 2 was available. In 2.1, options e 2 and 3 are possible.

Changing the behavior to be options 1 and 2 can be done for 2.2. I don't believe we can do the same for 2.1 as I don't consider this a bug as the default behavior in 2.1 matches that of 2.0.

[pracine]

What about exclude_nodata_value in other functions like ST_DumpAsPolygons()? Does it follow this behavior or same as ST_PixelAsPolygons()?

[dustymugs]

In most functions, exclude_nodata_value does as the parameter name implies. In 2.0, ST_DumpAsPolygons() didn't have the exclude_nodata_value parameter. It always excluded NODATA pixels.

ST_PixelAsPolygons() always included NODATA pixels in the output…

[alfred]

I have been using this workaround for now, by checking if the value is equal to the NODATA value.

In my case i have rasters where only small portions contain data and i am attempting to iterate the areas that have data and set values.

The problem with that is that it increases the query time significantly by going over all the pixels (can be thousands of those), even that i care only about the ones that have data only.

I was hoping that now that this function is implemented in C that this will provide boost in the speed, which it seems that it does in general, but not for this case.

I still think that having the "exlude_nodata_value" , should NOT return those pixels.

Having it set as false should return all the pixels and the values can still be checked with a WHERE clause. I don't see much benefit of having it set to NULL as in any case there needs to be a WHERE clause and the same can checked for being equal to ST_BandNoDataValue , but may be nice option in some cases.

[dustymugs]

I'll change the behavior for 2.2 unless robe indicates I can make the change for 2.1.

[dustymugs]

The problem with that is that it increases the query time significantly by going over all the pixels (can be thousands of those), even that i care only about the ones that have data only.

It will always go over every pixel. How else are you to know what the value of a pixel is?

I was hoping that now that this function is implemented in C that this will provide boost in the speed, which it seems that it does in general, but not for this case.

It was always in C, even in 2.0.

[alfred]

Replying to dustymugs:

The problem with that is that it increases the query time significantly by going over all the pixels (can be thousands of those), even that i care only about the ones that have data only.

It will always go over every pixel. How else are you to know what the value of a pixel is?

I was hoping that now that this function is implemented in C that this will provide boost in the speed, which it seems that it does in general, but not for this case.

It was always in C, even in 2.0.

I am sorry , what i mean is that if you do not believe that this check done in C is going to be faster than doing a SQL WHERE check, then probably there is no point in having this parameter (exclude_nodata_value).

[dustymugs]

My bad. ST_PixelAsPolygons was a pl/pgsql function in 2.0.

[dustymugs]

After chatting with robe2 on IRC, the decision was made to treat this as a bug. So, 2.1.1 it is.

[dustymugs]

Fixed at r11897 for 2.1. Fixed at r11898 for 2.2

[alfred]

Thank you! That is now an exponential increase in speed!

[robe]

exponential! — how exponential are we talking about?