Opened 5 years ago

Closed 3 months ago

#2263 closed task (fixed)

[raster] Audit usage of serialize/deserialize vs wkb/hexwkb

Reported by: Bborie Park Owned by: Bborie Park
Priority: high Milestone: PostGIS 2.5.0
Component: raster Version: trunk
Keywords: Cc:

Description

An audit needs to be done to delineate the usage of serialize/deserialize and wkb/hexwkb. At the present time, serialize/deserialize and wkb/hexwkb are the same and has possibly caused the blind usage of wkb/hexwkb when serialize/deserialize is appropriate and vice versa.

The key difference between serialize/deserialize and wkb/hexwkb is:

  1. serialize/deserialize: Internal storage format. The output seen from a query like the following should be of this format.
SELECT rast FROM mytable
  1. wkb/hexwkb: Transport format. This format is meant for use by and must be explicitly called for by clients. To get wkb/hexwkb output, the clients must use ST_AsBinary().

The rational for this is so that in the future, these two formats can deviate from each other (particularly the internal storage format).

Change History (9)

comment:1 Changed 5 years ago by Bborie Park

I'm expecting this audit to take place for PostGIS 2.2.

comment:2 Changed 5 years ago by Bborie Park

Milestone: PostGIS FuturePostGIS 2.2.0

comment:3 Changed 3 years ago by Bborie Park

Milestone: PostGIS 2.2.0PostGIS Future

comment:4 Changed 9 months ago by robe

Milestone: PostGIS FuturePostGIS Fund Me

Milestone renamed

comment:5 Changed 3 months ago by Bborie Park

Milestone: PostGIS Fund MePostGIS 2.5.0

The functions and what they're currently doing...

/*
	RASTER INPUTS
*/

-- expects input to be Hex WKB
CREATE OR REPLACE FUNCTION raster_in(cstring)
    RETURNS raster
    AS 'MODULE_PATHNAME','RASTER_in'
    LANGUAGE 'c' IMMUTABLE STRICT _PARALLEL;

-- TODO: does not exist yet
CREATE OR REPLACE FUNCTION st_rastfromwkb(raster)
    RETURNS bytea
    AS 'MODULE_PATHNAME', 'RASTER_fromWKB'
    LANGUAGE 'c' IMMUTABLE STRICT _PARALLEL;

-- expects output to be HEX WKB
CREATE OR REPLACE FUNCTION raster_out(raster)
    RETURNS cstring
    AS 'MODULE_PATHNAME','RASTER_out'
    LANGUAGE 'c' IMMUTABLE STRICT _PARALLEL;

/*
	RASTER OUTPUTS
*/

-- expects output to be WKB
CREATE OR REPLACE FUNCTION bytea(raster)
    RETURNS bytea
    AS 'MODULE_PATHNAME', 'RASTER_to_bytea'
    LANGUAGE 'c' IMMUTABLE STRICT _PARALLEL;

-- TODO needs better/descriptive name
-- expects output to be WKB
CREATE OR REPLACE FUNCTION st_asbinary(raster, outasin boolean DEFAULT FALSE)
    RETURNS bytea
    AS 'MODULE_PATHNAME', 'RASTER_to_binary'
    LANGUAGE 'c' IMMUTABLE STRICT _PARALLEL;

-- TODO: does not exist yet
CREATE OR REPLACE FUNCTION st_ashexwkb(raster, outasin boolean DEFAULT FALSE)
    RETURNS text
    AS 'MODULE_PATHNAME', 'RASTER_asHexWKB'
    LANGUAGE 'c' IMMUTABLE STRICT _PARALLEL;

comment:6 Changed 3 months ago by Bborie Park

Status: newassigned

comment:7 Changed 3 months ago by Bborie Park

After finishing the audit, the conclusion is that we're best not separating internal serialization structure and WKB at the PostgreSQL level. Instead, more documentation in the code and exposing more functions for input and output should be satisfactory to make things obvious for end-users

comment:8 Changed 3 months ago by Bborie Park

Raster Inputs

-- expects input to be WKB
CREATE OR REPLACE FUNCTION st_rastfromwkb(bytea)
    RETURNS raster
    AS 'MODULE_PATHNAME', 'RASTER_fromWKB'
    LANGUAGE 'c' IMMUTABLE STRICT _PARALLEL;

-- expectes input to be Hex WKB
CREATE OR REPLACE FUNCTION st_rastfromhexwkb(text)
    RETURNS raster
    AS 'MODULE_PATHNAME', 'RASTER_fromHexWKB'
    LANGUAGE 'c' IMMUTABLE STRICT _PARALLEL;

Raster Outputs

-- Availability: 2.5.0
-- expects output to be WKB
-- ST_AsWKB() is functionally identitical to bytea()
-- kept separate as bytea(raster) is for casting 
CREATE OR REPLACE FUNCTION st_aswkb(raster, outasin boolean DEFAULT FALSE)
    RETURNS bytea
    AS 'MODULE_PATHNAME', 'RASTER_asWKB'
    LANGUAGE 'c' IMMUTABLE STRICT _PARALLEL;

-- Changed: 2.5.0 change logic to use ST_AsWKB. Deprecate in 2.6.0
-- expects output to be WKB
CREATE OR REPLACE FUNCTION st_asbinary(raster, outasin boolean DEFAULT FALSE)
    RETURNS bytea
		AS $$ SELECT @extschema@.ST_AsWKB($1, $2) $$
    LANGUAGE 'sql' IMMUTABLE STRICT _PARALLEL;

-- Availability: 2.5.0
-- TODO: does not exist yet
-- expect output to be Hex WKB
CREATE OR REPLACE FUNCTION st_ashexwkb(raster, outasin boolean DEFAULT FALSE)
    RETURNS text
    AS 'MODULE_PATHNAME', 'RASTER_asHexWKB'
    LANGUAGE 'c' IMMUTABLE STRICT _PARALLEL;

-- used to cast raster -> bytea
CREATE OR REPLACE FUNCTION bytea(raster)
    RETURNS bytea
    AS 'MODULE_PATHNAME', 'RASTER_to_bytea'
    LANGUAGE 'c' IMMUTABLE STRICT _PARALLEL;

comment:9 Changed 3 months ago by Bborie Park

Resolution: fixed
Status: assignedclosed

Closing with commentary in the code as of r16385

Note: See TracTickets for help on using tickets.