WFS: posting doesn't always work

[fsteggink]

Hi,

I'm developing a WFS client, which I tested with multiple servers. I discovered that there were some problems with using HTTP POST to MapServer. (The default HTTP method of my client is GET, which works fine.) I've started a discussion on the MapServer users list. More information can be found there. I haven't installed MapServer myself, but was using public WFS servers on the internet.

Here are the results of one particular server (quoted from my mail): [quote] Well, it turned out that it depends on the presence of carriage returns in the post data if it will succeed or not. (The service URL is ​http://map.ns.ec.gc.ca/MapServer/mapserv.exe?map=/mapserver/services/envdat/config.map and the version is 4.8.3.)

Posting this will succeed (all on one line):

<?xml version="1.0" encoding="iso-8859-1"?><wfs:DescribeFeatureType version="1.0.0" service="WFS" xsi:schemaLocation="http://www.opengis.net/wfs ../wfs/1.0.0/WFS-basic.xsd" xmlns:wfs="http://www.opengis.net/wfs" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><wfs:TypeName>wfs:envirodat</wfs:TypeName></wfs:DescribeFeatureType>

Posting this will not:

<?xml version="1.0" encoding="iso-8859-1"?> <wfs:DescribeFeatureType version="1.0.0" service="WFS"
xsi:schemaLocation="http://www.opengis.net/wfs
../wfs/1.0.0/WFS-basic.xsd" xmlns:wfs="http://www.opengis.net/wfs"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <wfs:TypeName>wfs:envirodat</wfs:TypeName>
</wfs:DescribeFeatureType>

The post data is not URL encoded. And posting the successful post body to ​http://map.ns.ec.gc.ca/envdat/map.aspx gives the infamous "no template provided" error. quote This says two things.

1. There is a problem with the online resource URL's. (This is the last URL mentioned, and the online resource URL's are the URL's which are supposed to be used, instead of the base URL which is used for the initial GetCapabilities request only.) I found that multiple servers had this problem.

2. It apparently matters if there are carriage returns and/or line feeds in the XML data. Very strange. (If they are present, the service times out, and finally returns a 502 error after five minutes.)

The other services I have tested are:

• ​http://www.bsc-eoc.org/cgi-bin/bsc_ows.asp (MapServer 4.8.1);
• ​http://wfs.alaskamapped.org/inventory (MapServer 5.0 RC 2);
• ​http://excise.pyr.ec.gc.ca/cgi-bin/mapserv.exe?map=/LocalApps/Mapsurfer/PYRWQMP.map (MapServer 4.8.1);
• ​http://www2.dmsolutions.ca/cgi-bin/mswfs_gmap (MapServer 4.6.0).

Only the last service worked correctly with post data.

Since I don't have any MapServer experience (apart from testing my client against the WFS service), it is very hard to tell what the problem(s) is/are. Some people suggest that it could be a configuration / wrapper issue, but as I showed, there is also an issue with handling the data itself (the linebreaks). As I said, more information can be found in the e-mail thread, but this ticket contains the latest and most succint information.

[dmorissette]

There are multiple issues in this ticket:

1- The carriage return issue was fixed in MapServer 4.8.4 and 4.10.0 (ticket #1768)

2- You listed ​http://wfs.alaskamapped.org/inventory as having the carriage return problem, but it runs MapServer 5.0RC2 in which this should be fixed. Can you please double check that the error this server produces is indeed related to the carriage returns and not to something else?

3- You mention a "problem with onlineresource URLs". I'm not sure to follow you on this one. Can you please be more specific?

4- The ​http://map.ns.ec.gc.ca/envdat/map.aspx issue is a server configuration issue outside of MapServer's control since map.aspx doesn't support proper forwarding of POST request data. However, this server was very likely configured following the advice from section 2.4 of the WMS Server HOWTO at ​http://mapserver.gis.umn.edu/docs/howto/wms_server/#more-about-the-online-resource-url so this should be considered a documentation issue. I have added a comment about this at the bottom of this document. Hopefully they'll pick this up when the doc is updated.

[fsteggink]

Here are my replies:

1. I should have been a bit clearer, but the main issue is actually the "No template provided" error. This is what my first e-mail yesterday was about. The post is focuses a bit too much on the carriage return issue. Both are POST related issues, so for me it is a bit difficult to see if it is the same issue, or two different issues. Anyways, great to hear that it is solved. Now, if users only upgrade their versions sooner, it would be a lot easier for WFS clients :)

2. See also previous point. This server also has the "No template provided" error, and that is also true for the other ones, except for the one hosted at DM solutions. I haven't checked these for the carriage return issue.

3. The onlineResource URL in the Capabilities document (in wfs:Capability/wfs:Request/wfs:<requestname>/wfs:DCP/wfs:HTTP/wfs:<Get/Post>@onlineResource for WFS 1.0.0) specifies for each available WFS request and HTTP method the actual URL which should be used by the client. So, the URL which is used to do the initial GetCapabilities request should not be used further, unless the onlineResource URL's are the same. When I use the onlineResource URL's, as intended by WFS, I'm getting the "No template provided" error. That is what I meant with "problem with onlineResource URL's".

4. If it's a documentation issue, all the better :) I guess it will take a while before it is picked up.

I was wondering, what happens with the onlineResource URL's if MapServer is installed at another server, but the old map file is used? Or if the map file is renamed / moved itself? Will this be automatically reflected in the onlineResource URL's? I can imagine that this is often forgotten.

Hopefully things are clearer now.

[dmorissette]

Thanks for exercising those servers and all your work helping to identify the issues.

Here are more comments relative to the 4 points above:

1- Solved.

2.1- The reason why the DM Solutions server works even if it's older than 4.8.4 is that it runs on a Linux server and the carriage return issue happens only on Windows servers (not on Linux). The DM Solutions server also doesn't get the "No template provided" error because the "mswms_gmap" wrapper script *DOES* forward the POST data properly.

2.2- I'm pretty sure that all servers that get the "No template provided" on POST requests get that because they use a wrapper script for their onlineresource that doesn't forward POST data properly (as explained above in the comment about map.aspx documentation issue).

3. Your explanation makes sense and I agree with your interpretation of the spec. If the servers fix their ASP wrapper script to support POST then the advertized onlineresource should be valid and POST will work.

4. Hopefully the maintainers of those servers noticed the mailing list thread. :)

5- With respect to your last question, are you asking what happens with the onlineresource in the GetCapabilities if a MapServer onlineresource contains the map=... parameter and the mapfile location changes on that server? There are two possible scenarios:

Senario 1 - If the server admin lets MapServer determine the onlineresource automatically then the new mapfile path should automatically be reflected in the GetCapabilities output.

Scenario 2 - The server admin has the possibility of overriding the onlineresource value in the MapServer config (in MapServer lingo this is done in the ows_onlineresource mapfile metadata). This is mostly useful when dealing with servers behind firewalls or proxy'd servers. If the admin explicitly set ows_onlineresource and change the mapfile path then they also need to update their ows_onlineresource value to reflect the new mapfile path. The value of ows_onlineresource is pasted in the capabilities output without any checking.

Finally, I don't think there is anything left to fix in this bug in MapServer (other than the docs issue for the ASP wrapper but a comment has already been added to the docs for this), so I'll close this ticket now.