|Version 35 (modified by robe, 2 years ago)|
PostGIS Garden Test
The PostGIS Garden Test is a suite of SQL statements designed to stress test the PostGIS library within the PostgreSQL environment. It is a test generated from the PostGIS documentation reference.xml (PostGIS Reference section of manual) that tries to test every documented function against every kind of geometry supported by PostGIS as well as common issues like handling of nulls, empty geometries etc. The main objective of the tests is to try to catch bugs that will crash the server. It is also useful for monitoring odd behavior as well as a regression test against prior versions of the software. To test against prior versions, build a test from the prior version reference.xml and test against both the old and the new version on the same server and then do a diff between the two. With postgis-1.4, postgis-1.5, and postgis-2.0 this is much easier to do as you can run both versions on the same server in different databases.
To build the garden test you need an xsltproc (the same that is used to build the docs or some other .xsl processor). On windows you can download precompiled binaries from http://www.zlatkovic.com/pub/libxml/ and its part of the libxslt package. You may also need to download the iconv and zlib packages at same site and extract bin/.dlls in same folder as your xsltproc.exe.
The (reference.xml or postgis.xml) and (postgis_gardentest.sql.xsl, raster_gardentest.sql.xsl (introduced in PostGIS 2.0) located in the doc and doc/xsl folders are both needed to build the tests.
NOTE: These tests are equivalent to a monkey testing the software so a lot of tests fail. It's good in a sense as a monkey will stuff a geometry, geography or raster in any function that takes that and as such will test the system as human monkeys are bound to do.
To Build the script
Version PostGIS 2.0+
The latest version includes logic that will during the tests - create two tables in the test database and populate them for later inspection:
- postgis_garden_log: This contains a record for each test and logs both the start, end times and the sql statement that was run. columns are: logid- an autonumber, log_label - a short description, spatial_class - (will contain geometry or geography), func - name of function being tested, g1 - type of geometry of first geometry, g2 - type of geometry of second or null if a single geometry function, log_start - start time (timestamp), log_end - end time (timestamp), log_sql - the SQL statement that was run.
If the test crashes before completion, the record with the largest logid will tell you the query that crashed the server.
- postgis_garden_log_output -- this outputs the query results for queries that return something in xml format. It uses the built-in query_to_xml function that has existed since PostgreSQL 8.3 to do that.
- logid -- you can join this with the postgis_garden_log table to get the descriptive detail and SQL for the test.
- log_output - an xml field that contains the query_to_xml output of the query.
- There is also a companion for raster testing: raster_gardentest.sql.xsl. The raster one doesn't
currently create logging tables but will soon.
-- to run: -- create the tests There is an unfortunate dependency on postgis_agg_mm.xml which is really not needed just create a dummy blank file called postgis_agg_mm.xml in the trunk/doc to get around this Then:
xsltproc -o geo_torturetest.sql trunk/doc/xsl/postgis_gardentest.sql.xsl trunk/doc/postgis.xml xsltproc -o rast_torturetest.sql trunk/doc/xsl/raster_gardentest.sql.xsl trunk/doc/postgis.xml
-- run them
psql -p 5432 -U postgres -d postgres -c "DROP DATABASE testpostgis;" psql -p 5432 -U postgres -d postgres -c "CREATE DATABASE testpostgis;" psql -p 5432 -U postgres -d testpostgis -c "CREATE LANGUAGE plpgsql;" psql -p 5432 -U postgres -d testpostgis -f postgis.sql psql -p 5432 -U postgres -d testpostgis -f spatial_ref_sys.sql psql -p 5432 -U postgres -d testpostgis -f rtpostgis.sql psql -p 5432 -U postgres -d testpostgis -f geo_torturetest.sql > geo_torturetest_results.txt psql -p 5432 -U postgres -d testpostgis -f rast_torturetest.sql > rast_torturetest_results.txt
One nice benefit about logging the query and the output to a table, is that it's easier to inspect (and you can conceivably (thought haven't tried yet) do a join between two log output tables by logid to compare results from different versions of postgis) and you can also rerun a subset of the queries for closer inspection by just writing an sql statement something like below -- which will test all tests for the ST_3DDistance function that involve points and linestrings
SELECT logid, log_label, query_to_xml(log_sql, false,false,'') As result, log_sql FROM postgis_garden_log WHERE func = 'ST_3DDistance' AND (g1 IN('POINTZ', 'MULTIPOINTZ', 'LINESTRINGZ') OR g2 IN('POINTZ', 'MULTIPOINTZ', 'LINESTRINGZ') ) ) ;
You can even do even more cool things like use pgbench to simulate users with a transaction script that looks something like the below script that will run 10 function queries on functions that start with ST_ that have succeeded in the past.
BEGIN; SELECT query_to_xml(log_sql) FROM postgis_garden_log WHERE func LIKE 'ST_%' and log_end IS NOT NULL ORDER BY random() LIMIT 10; END;
Version PostGIS 1.5+
There is an unfortunate dependency on postgis_agg_mm.xml which is really not needed just create a dummy blank file called postgis_agg_mm.xml in the trunk/doc to get around this Then:
xsltproc -o torturetest.sql branches/1.5/doc/xsl/postgis_gardentest.sql.xsl branches/1.5/doc/postgis.xml
Below is a basic script to test tests and run them (if you are running from dos, change the slashes or just run everything from same folder:
psql -p 5432 -U postgres -d postgres -c "DROP DATABASE testpostgis;" psql -p 5432 -U postgres -d postgres -c "CREATE DATABASE testpostgis;" psql -p 5432 -U postgres -d testpostgis -c "CREATE LANGUAGE plpgsql;" psql -p 5432 -U postgres -d testpostgis -f postgis.sql psql -p 5432 -U postgres -d testpostgis -f spatial_ref_sys.sql psql -p 5432 -U postgres -d testpostgis -f torturetest.sql > torturetest_results.txt
If you want it to output the actual query that is being tested, then use the -a command like so
psql -p 5432 -U postgres -d testpostgis -f torturetest.sql -a > torturetest_results.txt
If you want to include timing information - you can edit the generated torturetest.sql and add a
Version PostGIS 1.4
xsltproc -o torturetest.sql branches/1.4/doc/xsl/postgis_gardentest.sql.xsl branches/1.4/doc/reference.xml
Testing subset of functions
A companion to the full postgis_gardentest.sql.xsl is the postgis_gardentest_subset.sql.xsl. This version skips the table creation battery of tests and allows you to specify a subset of functions to test.
This test is useful to test out new functions introduced or just test functions that have changed from prior versions to make sure they are behaving as expected. It makes the output file much shorter and easier to scan.
It will only test functions whose ids are contained in the inputfninclude parameter. NOTE this currently causes a bit of overtesting since ST_MakeLine and ST_M will match ST_MakeLine. Below is an example use.
xsltproc --param inputfninclude '"ST_Collect ST_Distance ST_DWithin ST_GeomFromGML ST_Length"' -o torturetest_subset.sql trunk/doc/xsl/postgis_gardentest_subset.sql.xsl trunk/doc/reference.xml psql -p 5432 -U postgres -d postgres -c "CREATE DATABASE testpostgis;" psql -p 5432 -U postgres -d testpostgis -f postgis.sql psql -p 5432 -U postgres -d testpostgis -f spatial_ref_sys.sql psql -p 5432 -U postgres -d testpostgis -f torturetest_subset.sql > torturetest_results.txt psql -p 5432 -U postgres -d postgres -c "DROP DATABASE testpostgis;"