Changes between Version 4 and Version 5 of DevWikiPostGISCoding
- Timestamp:
- 12/09/10 13:38:51 (14 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
DevWikiPostGISCoding
v4 v5 5 5 == PostgreSQL is the Man Behind the Curtain == 6 6 7 All memory allocation in the ./liblwgeom directory should be done with lwalloc/lwfree/lwrealloc and in the ./postgis directory should be done with palloc/pfree/repalloc. Why so? the core, memory management in PostGIS does not matter because PostgreSQL actually handles all memory in a "heirarchical memory manager". That means PostgreSQL is the one calling the actual system malloc, and your palloc calls are managed by PostgreSQL inside larger pages ("contexts") that it mallocs. Each time a function is called, PostgreSQL creates a new memory context and all palloc/pfree calls happen in that context; when the function call is complete, the whole context is discarded, which means any extra memory you failed to free is discarded too. Basically all your memory management is for naught, because PostgreSQL will clean up after the end of the function call.7 All memory allocation in the ./liblwgeom directory should be done with lwalloc/lwfree/lwrealloc and in the ./postgis directory should be done with palloc/pfree/repalloc. Why so? At the core, memory management in PostGIS does not matter because PostgreSQL actually handles all memory in a "heirarchical memory manager" behind the scenes. That means PostgreSQL is the one calling the actual system malloc, and your palloc calls are managed by PostgreSQL inside larger pages ("contexts") that it mallocs. Each time a SQL function is called, PostgreSQL creates a new memory context and all your palloc/pfree calls happen in that context; when the SQL function call is complete, the whole context is discarded, which means any extra memory you failed to free (whoops!) is discarded too (phew!). Basically all your memory management is for naught, because PostgreSQL will clean up after the end of the function call. 8 8 9 The lwalloc/lwfree calls in liblwgeom are themselves palloc/pfree calls when made within the context of the PostgreSQL engine. However, they can also be called outside of PostgreSQL, for example in the shp2pgsql and pgsql2shp utilities: in those cases they are direct calls to malloc/free. So memory management matters more in ./liblwgeom than in ./postgis. In ./liblwgeom there might not be someone clean ly upbehind you.9 The lwalloc/lwfree calls in liblwgeom are themselves palloc/pfree calls when made within the context of the PostgreSQL engine. However, they can also be called outside of PostgreSQL, for example in the shp2pgsql and pgsql2shp utilities: in those cases they are direct calls to malloc/free. So memory management matters more in ./liblwgeom than in ./postgis. In ./liblwgeom there might not be someone cleaning up memory behind you. 10 10 11 It's important not to abuse the PostgreSQL memory manager though, because sometimes our GIS calls use up a lot of memory inside just one function call. Below are some PostgreSQL MACROs and ./liblwgeom functions that are useful for memory management and/or just confusing in and of themselves.11 However, it is important not to abuse the PostgreSQL memory manager, because sometimes our GIS calls use up a lot of memory inside just one function call. Try to clean up as if you were not working inside a memory manager. Below are some PostgreSQL MACROs and ./liblwgeom functions that are useful for memory management and/or just confusing in and of themselves. 12 12 13 13 == PG_FUNCTION_INFO_V1(functionname) == … … 20 20 The actual arguments to a PostgreSQL function are various function contexts and database internals that we don't want to care about. All that stuff is hidden in this macro, and we use the PG_GETARG macros later on to retrieve the information we really want. 21 21 22 22 23 == PG_GETARG == 23 24 24 25 Most of the PG_GETARG calls are pretty self explanatory. There's the PG_GETARG part, the part that declares the data type you are retrieving, and the argument number you are retrieving. So, PG_GETARG_INT32(0) gets the first argument as an integer. 25 26 27 26 28 == PG_DETOAST_DATUM(PG_GETARG_DATUM()) == 27 29 28 30 Datum? Every piece of information in the database is passed around as a Datum, which is a de-natured pointer. For big objects, like geography objects, the datum itself might not point directly to the object, it might point to a "TOAST tuple" which in turn points to where the data is stored. In order to access the whole object, we need to "de-TOAST" it, hence we first get the datum number for our argument object, then de-toast it into a pointer. The pointer is untyped, so you will usually see this macro call in conjunction with a (TYPE*) cast to the appropriate pointer type. 31 29 32 30 33 == VARLENA == … … 47 50 }}} 48 51 52 53 == LWGEOM vs PG_LWGEOM vs GSERIALIZED == 54 55 PG_LWGEOM and GSERIALIZED are both PostgreSQL varlena types. They have 4 bytes of metadata about size, then a payload. The payload of the PG_LWGEOM is descrived in the ./postgis/SERIALIZED_FORM document, and the payload in the GSERIALIZED is described in ./liblwgeom/gserialized.txt. Basically they are serializations of a geometry, similar to well-know-binary. 56 57 LWGEOM is a struct, with much more internal structure than the varlena byte strings. It has a type number, some flags, it has coordinates that are represented by a pointer(s) to POINTARRAY structs, or if it is a collection type it has sub-geometries represented by more pointers to LWGEOM structs. The ./liblwgeom directory is a library for working with LWGEOM structs. 58 59 You can get LWGEOM from PG_LWGEOM using pglwgeom_deserialize(), and you can get a PG_LWGEOM from an LWGEOM using pglwgeom_serialize(). 60 61 You can get LWGEOM from GSERIALIZED using lwgeom_from_gserialized() and you can get a GSERIALIZED from an LWGEOM using gserialized_from_lwgeom(). 62 63 Once you have instantiated an LWGEOM, you should remember to free it with lwgeom_free(). In the case where you want to free the LWGEOM struct and bounding box, but but leave the sub-geometries or POINTARRAYs intact, use lwgeom_release(). 64 65 66 == PG_FREE_IF_COPY() == 67 68 As noted above, when you PG_DETOAST_DATUM, you create an in memory representation of the varlena. Sometimes this is a copy and sometimes it is a direct pointer to the original tuple. You can free the copy with PG_FREE_IF_COPY(varlena_pointer, argumentnumber). So, if you detoasted argument zero into a varlena named foo, you'd call PG_FREE_IF_COPY(foo, 0) before exiting the function. See the example below. 69 70 49 71 == An Example == 72 73 Note the order in which things are allocated and freed. 74 75 # We get our pointers to the varlena objects 76 # We deserialize them into LWGEOM 77 # We do our work 78 # We serialize our result back to a varlena 79 # We free our LWGEOM 80 # We conditionally free our input varlena 81 # We return 50 82 51 83 {{{ … … 81 113 82 114 /* First free the LWGEOMs you used */ 115 lwgeom_free(theline); 83 116 lwgeom_free(lwgeom1); 84 117 lwgeom_free(lwgeom2);
