[WFS] Support for "Level 0 Profile of GML3 for WFS"

[dmorissette]

Bug 577 raised an interoperability issue between MapServer and ArcMap: ArcMap
just can't ingest anything but its own flavour (profile) of GML. There is not
much we could do about this specific bug at the moment, but there is hope: there
is a "Level 0 Profile of GML3 for WFS" discussion paper that's making its way at
OGC. 

If/when this is formally adopted and implemented by OGC members, we should
implement this profile in MapServer as well, and then we can all interoperate
much more easily with WFS servers.

Non-OGC members can find an older copy of this discussion paper (from May 2004)
in the public docs section of the opengis.org website. There will be more work
on this by OGC members during the fall, so we can expect more news on this in
January 2005 after the next OGC TC meetings.

[sdlime]

I think we'll run into these flavours increasingly overtime. For example, at the
DNR we've got a grant from the FGDC to implement a few "framework" layers (e.g.
governmental boundaries) using WFS. They have their own distinct flavour of GML.
The geometries are not the problem, it's the attributes. I don't believe the
vanilla GML that MapServer kicks out will be enough for many of the framework
themes. What about the idea of "templating" GML or more generically WFS output?
Much the same way as queries are handled now. That way you could construct very
complex GML (or other text-based formats) documents. One layer could potentially
have several templated output formats (plus MapServer vanilla GML 2.x/3.x), one
for ArcGIS, one for FGDC and so on if necessary. In nice feature of templates is
that you can basically transform the data on-the-fly, mapping attributes from
one model to another. For FGDC framework data we could probably then come up
with a base set of templates that anyone could apply. Just thinking out load...

Steve

[dmorissette]

I like the idea, but I don't see a simple way to make it happen. Just the
current query templating mechanism is too simplistic for that since we need to
deal with GML schemas (DescribeFeatureType) as well. Should we define a kind of
"attribute -> schema" mapping mechanism?  I guess I need to sleep on it.

[sdlime]

Another idea might be to allow users to transform our "generic" GML using 
xslt. There's a C library for doing just that (http://xmlsoft.org/XSLT/). 
Which stylesheet could be controled via metadata perhaps... Interesting topic.

Steve

[fwarmerdam]

Oh no, Steve is turning into an XML weenie!

[sdlime]

And I think XML mapfiles would be swell...

[tomkralidis]

My $0.02:

I think MapServer can offer the existing way of pushing out GML in the core
software.  As for profiles, etc., it would be great to inherently support these
as well.

We should also allow for custom output formats, much like the HTML templating is
done, for complex GML (3) patterns, etc.

Perhaps defining an OUTPUTFORMAT object:

OUTPUTFORMAT
  NAME GML3
  MIMETYPE "application/vnd.ogc.gml3"
END

Then, each LAYER object would have to have templates defined, i.e.:

LAYER
 NAME roads
 DATA data/roads
 FORMAT
  NAME GML3
  HEADER "roads_header.gml3"
  TEMPLATE "roads_body.gml3"
  FOOTER "roads_footer.gml3"
 END
END

...just thinking out loud...

[sdlime]

There are a couple of problems with templates:

1) With WFS you can limit the items returned, but templates make that tough 
unless you build in special parsing capabilities.

2) You can only have 1 set of templates per layer. Kinda sucks, cause you might 
like to have GML 2.1, GML 3.1, GML 3.1 (FGDC framework profile), GML 3.1 (works 
with ArcMap profile) and so on for one layer.

I could see doing something like:

OUTPUTFORMAT
  NAME "GML3, FGDC Hydrography Framework"
  DRIVER GML3
  STYLE "some XSLT stylesheet"
  MIMETYPE "application/vnd.ogc.gml3"
END

The style sheet would transform our vanila GML to the correct profile. Then for 
each layer you define what output formats you support. I know the WCS spec 
supports coverage level format lists. Does WFS?

Steve

[dmorissette]

Let's keep in mind that the topic of this bug is how to add support Level 0
profile of GML for WFS. If we want to talk about all sorts of flavours or GML
throught XSLT or another mechanism then perhaps we should create another more
generic bug and make it a blocker of the current bug.

Anyway, I think that replacing the current GML that MapServer produces with GML
level 0 would be a smart idea. Then we can add XSLT or whatever on top, but at
least, MapServer's core GML will correspond to some standard and won't be just
another flavour of GML. Based on my reading of the GML L0 discussion paper,
supporting this flavour instead of the current one should not change much, this
profile matches MapServer's simple-feature data model very well. Doing the
switch would just clean things up a bit and ensure that we generate some
well-known format of GML instead of just polluting the environment with
MapServer's own flavour of GML.

My 0.02$.

[dmorissette]

Created bug 950 about the generic profile mechanism and made it a blocker of
this bug.

[dmorissette]

Setting target milestone to FUTURE. The GML profile is not adopted yet.

[sdlime]

Dan: Do you have access to the documentation for the GML3 level 0 profile? I
assume it's behind the scenes someplace at OGC...

Steve

[dmorissette]

Actually, there is a slightly older version of the discussion paper on the
public OGC site: http://portal.opengis.org/files/?artifact_id=4347

(The most recent version that I aware of is from Aug. 2004 (04-061) and is not
public, but based on the revision history the changes would be very minor)

[sdlime]

Hi all: I need to beging working on this one. A few logistical questions before 
I begin:

- Should I simply a version parameter to the relevant functions in mapgml.c so 
that either 2.1 or 3.0L0 OR should the GML production functions be version 
specific with control which version happening in msGMLWriteQuery or 
msGMLWriteWFSQuery?

- mapgml.h exists but is unused. The GML functions are listed in mapows.h. 
Should we ditch mapgml.h altogether?

- in mapgml.c much of the generic GML code is conditionally compiled only if 
WMS or WFS servers are enabled. Makes sense for WFS and the msGMLWriteWFSQuery 
function, but not for WMS. GML has utility beyond WMS and should be available 
as an output format if PROJ.4 is available.

Thoughts/comments?

Steve

[sdlime]

The encoding functions in mapgml.c just need to be ported to GML 3.0. We get 
level 0 compliance by default since MapServer isn't dealing with funky geometry 
or attribute types. Working on that part now. My approach is to write format 
specific (e.g. GML2 vs. GML3) functions to encode a bounding box and a geometry 
(any GML3 geometry encoding will only use valid GML3L0 property types) so you 
have:

  gmlWriteBounds_GML2  gmlWriteBounds_GML2
  gmlWriteBounds_GML3  gmlWriteBounds_GML3

and then wrapper functions gmlWriteBounds and gmlWriteGeometry that are called 
by the full GML writing functions.

Steve

[sdlime]

The GML authors are *very* frustrating, friggin' keep changing their minds...

What should our target GML version be 3.0 or 3.1? Seems like we should be 
shooting for 3.1 since the GML3-Level 0 document hints at it. Yet it's there 
are many contradictions. For example, to encode a multi-line feature level 0 
limits you to a MultiLineString which of course is depricated in GML 3.1 in 
favor of a MultiCurve (as is a MultiPolygon for MultiSurface). Plus there is no 
MultiPoint geometry listed in level 0. 

Anyway seems like a good bit of change between 3.0 and 3.1. Any idea when 3.1 
will be out and the level 0 profile will be finalized? Anyone?

Steve

[bartvde@…]

Hi Steve,

I don't know if you have access to the OGC portal. But the latest version of the
GML 3 level 0 profile is (although it is pending and has not been adopted yet):

04-061  	  Level 0 Profile of GML3 for WFS  	Panagiotis (Peter) A. Vretanos  
547 KB   	doc  	 2004/08/23 1:21 PM

The latest GML 3 document on the portal is 3.1.1, and there have been a few bug
fix releases on the 3.1.1 schema already, the last version is:

 04-092r2  	  GML 3.1.1 Schema (bugfix release) (XML Schema documents)  	Simon
Cox  	216.85 KB   	zip  	 2004/12/13 04:35 AM

Best thing is probably to send an e-mail to the editor of the Level 0 Profile
document, "Panagiotis (Peter) A. Vretanos" <pvretano at cubewerx.com>.

Bart

[sdlime]

I've got access via the UMN and already have the 8/23 document. Had not seen 
the 3.1.1 docs though. I'll try contacting Peter...

Steve

[dmorissette]

Do we really need to continue to support the old GML2 format? My initial idea
was to ditch the old/custom GML2 that we used to have (mapgml.c), and to replace
it with GML3L0 compliant output. That would include switching the
DescribeFeatureType requests to the new GML3L0 format as well. I realize that
this may break compatibility for clients that rely on the old/custom GML2 format
that we used to produce, but since we used to produce a custom flavour of GML
anyway and would replace it with a well-known and documented format, I think
this change would be for the best.

About your other questions: I guess it would make sense to ditch mapgml.h and to
enable GML output independently of the WMS/WFS interfaces as you suggest.

[sdlime]

GML2 looks to be required for WFS 1.0.0 so I think we need to keep it.

Steve

[sdlime]

I've enabled WFS in 4.5 to provide 2 output formats GML2 and GML3. The GML3
encoding conforms to the most recent version of the GML3L0 proposals I've seen
from Peter Vretanos.

Now on to DescribeFeatureType...

Steve

[sdlime]

The thread is so long and the topic so broad I'm going to close this and open 
new, more specific bugs as necessary related to GML3L0 and application schema...

Steve