query function restructuring

[assefa]

Here is an e-mail from Frank Warmerdam on thei issue :

On a vaguely related note, the current query interface seems to make a
pass generating the 'item ids' for all the features which match the
query.  Then when the query is returned another pass is made fetching the
shapes by id.  With shapefiles this isn't particularly expensive, though it
does presumably result in the features being parsed from the binary twice.

But for some OGR supported formats fetching "by id" can be very expensive.

If we are restructuring the query mechanism what are the chances of reviewing
this whole two pass approach?  Since WFS is basically always using the
query mechanism it seems this is a significant performance issue for some
WFS setups.  Even for "good formats" there is a bunch of overhead.

[sdlime]

There is another bug in the system related to this. Don't remember the number 
offhand, but it was related to PostGIS. Shapefiles are so nice and simple...

I've not worried too much about query speed because for the most part a query 
was always the end product of an interaction with an application (i.e. zoom 10 
times and then query). That does seem to be changing, but you don't hear people 
complaining too much.

The change would be significant but would not require a major rewrite. I guess 
we would need to maintain a list of shapes within each layer and would need 
utility functions to add/remove/replace/retrieve shapes in that cache. (I 
should add that I would probably advocate using the same caching mechanism with 
the labelcache so that more advanced placement could be done, such as text 
splining). There is already a set of shape-based linked list routines. My worry 
when the query stuff was originally put together was that for large results 
sets memory could become an issue. Not for a single query (although I suppose 
that could easily happen) so much but in cases where multiple instances of 
MapServer were being executed at the same time. Using IDS mitigated that 
problem. Any ideas how to deal with excessive memory usage?

Steve

[sdlime]

We'll be filling other tickets when an RFC gets written to deal with one-pass queries. Marking this as WONTFIX just to clean up some of these old ones.

Steve