Ticket #460 (assigned defect)

Opened 5 years ago

Last modified 4 years ago

Query Efficiency

Reported by: dblasby@refractions.net Assigned to: sdlime (accepted)
Priority: high Milestone:
Component: MapServer CGI Version: unspecified
Severity: normal Keywords:
Cc: bartvde@osgis.nl

Description 

1. Noone seems to know how the query (itemquery, itemnquery) mechanics actually
work.  Some investigation and documentation are needed.

2. The query method appears to be (a) do a query-by-rectangle (with filter) to
get record#s, then (b) repeatedly request each feature by record number.  In
order for this to be efficient, it assumes a O(1) get-feature-by-record-number
access.  This isnt true for all layer types.  Also, cacheing the results of the
query-by-rectangle (with filter) isnt generally possible because the columns
(items) havent been set yet in the layer object or by a call to LayerGetItems().

3. Why is there a separate query/draw mechanism?  Why doesnt mapserv just do a
normal draw-type request (query-by-rectangle (with filter)), then use the normal
 getNextShape() functions?  Its a maintainance issue to have each layer
implement two basic functions (draw and query) when query could be seamlessly
implemented by the draw mechanism.

C.F. mailing list discussion with subject "Re: [Mapserver-dev] Re:
[Mapserver-users] Mapscript drawQuery() on big PostGIS layers doesn't work
properly ?"

Daniel requested a bug report so this doesnt go away.

Change History

10/16/03 09:59:33 changed by dmorissette

  • cc set to steve.lime@dnr.state.mn.us.
Adding Steve to the CC

10/16/03 12:00:02 changed by sdlime

  • owner changed from morissette@dmsolutions.ca to sdlime.
1. I know how they work. What do you want know?

2. Which layer types don't support feature access by number? The only one can 
recall is Oracle Spatial but it's not clear. Mind you this is how drawing works 
too. Where queries differ is that they store IDs for processing later through 
the templates. This requires accessing them twice which is definitely 
inefficient but queries generally don't have to be as snappy. Part of the 
reason only IDs are cached is that large queries could eat up tons of memory. 
We could cache entire shapes and attributes as long as care is taken with large 
result sets. This makes saved queries very difficult, but actually outputing 
shapes as GML might be a possibility.

3. Cause it was a kludge and should be re-written. Queries can be much more 
complex then rect/filter searches (e.g. feature queries) so they have to work 
off a cache of some sort.

10/16/03 13:10:15 changed by sdlime

  • status changed from new to assigned.
  • cc set to morissette@dmsolutions.ca.
For SDE and shapefiles feature ID based access is quite fast (those were my 
models when writing that code). If that's not the case in general then I need 
to fix the issue. Caching entire shapes with attributes would be the answer. 
This would not be hard and would generally simplify life but would touch a lot 
of code so it would be time consuming.

Steve

11/18/04 14:03:00 changed by bartvde@osgis.nl

  • cc set to bartvde@xs4all.nl.