Need support for ITEM data types for WFS server

[dmorissette]

As part of the WFS enhancements, we need to be able to return proper attribute
data type information in DescribeFeatureType responses.  The current version of
MapServer treats all attributes (items) as strings and this bug is to discuss
any implications/objections of a change to support attribute data type
information in MapServer 4.1.


Here is what we propose:

1- Add a mechanism in MapServer to maintain field (item) name, data type, size,
and precision.  This would likely be a new itemDefnObj that would be set by
msLayerGetItems() instead of the array of field names that it currently produces.

2- Each data source driver would then have to be modified to return proper type
information.  For a start we should make all drivers return item data types as
"string" by default, and then we should file bugs for each driver maintainer to
update their driver when they have time.

3- For the time being, attribute values would still be carried as strings
internally.  We could switch to using a union similar to OGR's OGRField later
on, but for now we'll carry all values as strings internally to limit the amount
of changes to the code.

4- The WFS DescribeFeatureType will need to be extended to publish data types. 
Note that this implies that some rules related to those data types in XML will
have to be followed.

Any comments, objections, suggestions?

[sdlime]

Sounds reasonable to me, should be a fairly easy change to make...

Steve

[sdlime]

On a side note this would then allow us to do something that's been bugging me 
for years- format query results. Since everything is a string you can't format 
floating put output. It would great to be able to use attribute substitution 
strings like:

  [item name="AREA" escape="URL" format="%.2lf"]

One thing this might need is more effective way to look up attributes, so 
perhaps storing the itemDefObjs in a hash would be better then using an array.

Steve

[dmorissette]

Starting work on this now.

About using a hash to store the itemDefnObjs Steve, I don't think that should be
the main access mechanism since it's still more expensive than access by index
which we need to do sometimes.  So I think we would still store the idemDefnObjs
in an array, but we could add a hash on the side to quickly map item name to
itemDefn index.  Is that OK with you?

I've also created separate bug 474 for the formatting of query results since
it's out of scope of the current bug and we wouldn't want to lose this idea.

[sdlime]

As long as there is some fast way of doing name-based lookups I'll be happy.

[assefa]

Daniel what is the status on this. Do you intend to add this functionnality or 
should we set it to FUTURE.

[dmorissette]

I think it's too late for 4.4, I'll have to get back to this in a future
release. (setting target to FUTURE)

[dmorissette]

*** Bug 1638 has been marked as a duplicate of this bug. ***

[tomkralidis]

This would be a valuable addition, and would effectively eliminate stuff like #1903

[sdlime]

Right now you can set the data type for a given field in the gml_variable_type metadata the the WFS server will use that in schema construction. It's not automated in any fashion so this would still be a nice addition. It is a work around however, and you'd probably always want the ability to override the stored type (some data providers can't store proper types) anyway.

Steve

[bartvde]

Any chance of this getting some attention in the near future? Is there anyway I can help with this change?

[sdlime]

Bart: You're interested in the auto typing functionality correct? That's a relatively significant enhancement in spite of some of the early comments simply because of the number of native providers we have and the ammount of code that would need touching. Personally I think we'd need to create a new itemObj struct to hold name, index, type, ... and then tweak all providers to fill that out for each item. It really would simplify like in many ways.

It's big enough that I don't know who can bite it off as a pet project so funding might really help. Just my 2 cents.

Steve

[dmorissette]

I agree with Steve that this is one of those lots-of-work-nice-to-have features that have a hard time making anyone's priority list. Perhaps bug us again about this early in the 5.4 development cycle?

[dmorissette]

This came up again in ticket #2902. Let's try to get this in 6.0.