[WMS/WFS] Ability to specify a proxy for remote connections

[dmorissette]

We need the ability to specify a proxy when connecting to remote WMS/WFS servers.

This could possibly be added as layer metadata wms_proxy_*, but I wonder if we should instead create global configuration parameters inside the WEB object?

I never really worked in environments with proxies, so I wonder if it is safe to assume that when someone uses a proxy, then they use it for *all* their http connections and not only to get to specific hosts? i.e. is it a good idea to have a single global proxy setting in the web object in the mapfile or should this be local at the layer level?

[fwarmerdam]

I agree that this should be map wide, rather than layer specific.  

I am not sure it belongs in the WEB section though, since it relates to 
how remote requests are made, now how the local configuraton works.  

There are also a number of other global configuration options I would like
to add in the not-to-distant-future, including .map file driven overrides for
GDAL_DATA, and PROJ_LIB.   I'm not sure that these are related though. 

[bartvde@…]

The production environments I have been in would only need to have a global 
proxy setting in the MAP file, not a per layer proxy setting.

[franck.villoin@…]

Attention, we can imagine use Services WMS without proxy for certain layers in 
the Intranet, and others services WMS/WFS out of intranet use the proxy.
But I use only one the proxy in my production environment. 

[dmorissette]

Franck, what would happen in your case if you tried to access a WMS server from
the intranet via the proxy?  Would things still work?

[franck.villoin@…]

No, it does not function. 
If I pass by my proxy, the addresses of my Intranet are not solved.
By fixing the proxy in a permanent way in my navigator,
I do not see any more the sites of my Intranet.

Is it possible to define a global proxy parameter, and to make it active by 
layer if necessary?

Franck

[dmorissette]

Pushing to 4.4 release

[tjmitchell@…]

I also need this functionality due to some relocation of our servers.  Our 
proxy requires our username/password too!  Ugh.

Curl docs say:

"NOTE2: libcurl respects the environment variables http_proxy, ftp_proxy, 
all_proxy etc, if any of those is set."

Can this be used somehow as a quick workaround?

Tyler

[dmorissette]

The libcurl tutorial talks about those environment variables as well. They must
be lowercase, and the value is in the format
"[protocol://][user:password@]machine[:port]". They also note that the
"protocol://" part is simply ignored if present. (Proxies are always http).

Tyler, I think the best way to go is for you to try this and let us know if that
works.

[dmorissette]

Not in 4.4. Setting FUTURE target milsetone.

[bartvde@…]

I checked this by using SetEnv http_proxy in the Apache conf and all works fine!
There is also an option in curl to exclude certain domains if needed.

For my use this would be enough. I can't think of any situation in which you
want per layer proxy url's.

[elromero@…]

Hello,

I'm a new MapServer user and I have a problem. I want to call a layer from a 
remote WMS but I can't because I have a problem whith the proxy.
I have inserted the next line in httpd.conf file:
SetEnv http_proxy "http://isas.vivienda.local:8080"

And I have inserted in my map file:
LAYER # MODIS WMS map from JPL
    NAME         modis_jpl
    TYPE         RASTER
    OFFSITE      0 0 0
    STATUS       OFF
    CONNECTIONTYPE WMS
    DEBUG        ON
    CONNECTION "http://wms.jpl.nasa.gov/wms.cgi?"
    
    METADATA
      "wms_srs" "EPSG:4326"
      "wms_name" "modis"
      "wms_server_version" "1.1.1"
      "wms_format" "image/jpeg"      
      "wms_proxy_*" "isas.vivienda.local:8080"
    END

    PROJECTION
      "init=epsg:4326"
    END
  END # Modis WMS image ends here

But I only get a temporary file with an autentification proxy problem.

What am I doing bad? What more do I need?

Thak you,

Emilio

[dmorissette]

Emilio, this is a bug report, it is not the place to ask for support, I would
suggest that you send your question to the mapserver-users list. If it turns out
that better proxy support would solve your problem then the conclusions of the
list discussion can be added in support to this bug later on.
(Hint: also include a copy of the proxy error messages in your post to the list.)

[project10]

Initial implementation

[project10]

I've attached a patch which provides an initial implementation of the ability to specify proxying information, as well as HTTP authentication information for WMS layers. The implementation uses layer- and web-level metadata, as discussed on #mapserver.

The patch adds the following new metadata directives:

  'wms_proxy_host' '192.168.2.10'

• can be specified as a hostname, dotted-quad, with optional port component (e.g. '192.168.2.10:8080')

    'wms_proxy_port' '8080'
  

• if the port is not given above, can be specified with this metadata

    'wms_proxy_type' 'http'
  

• supported types: 'http', 'socks5' (case insensitive)

    'wms_proxy_auth_type' 'ntlm'
  

• supported types (case insensitive):
  • basic
  • digest
  • ntlm
  • any (the underlying http library picks the best among the options supported by the remote server)
  • anysafe (the underlying http library picks only safe methods among the options supported by the remote server)

      'wms_proxy_username' 'foo'
      'wms_proxy_password' '{FF88CFDAAE1A5E33}'
    

• msEncrypt-style string supported; also accepts "naked" strings

    'wms_auth_type' 'ntlm'
  

• same supported options as wms_proxy_auth_type

    'wms_auth_username' 'foo1'
    'wms_auth_password' '{FF88CFDAAE1A5E33}'
  

• msEncrypt-style string supported; also accepts "naked" strings

[sdlime]

I have applied the patches to the dev trunk. If folks could please test and report back I'd appreciate it. I'm moving this to a documentation bug at this point.

Steve

[dmorissette]

I just committed r7562 with fixes to the r7560 patch to make it compile cleanly, and added MS_ prefix to the proxy/auth type constants to avoid possible symbol collisions in the future (#571).

I don't have a proxy to test with, so I just made sure the build worked.

[dmorissette]

Doh! In the comment above I meant r7652 ... which fixes r7650 ... sorry!

And for those who might notice the new maplexer.c as part of the same commit, it was due to get into SVN separately, it is related to ticket #2634 (r7633) and got committed at the same time by accident.

[sdlime]

Replying to dmorissette:

Dan, thanks for the fixes! -Steve

[dionw]

allow build against curl < 7.10.7 (e.g. RHEL3)

[dionw]

Using CURLOPT_PROXYAUTH is only available starting with curl 7.10.7. To allow building against earlier versions (e.g. RHEL3 comes with 7.10.6) I've created the above patch.

Additionally, the patch introduces a warning when using Mapserver build with curl that doesn't support setting the proxy auth type, but nevertheless supplying the wms_proxy_auth_type metadata in the mapfile. Perhaps this should be an error.

[dmorissette]

Applied the patch to build against Curl 7.10.6 and older, and also log an error to warn the user but without aborting the execution. The error will look as follows and will be trappable through the error stack, or visible in the DEBUG output:

[Tue Jul  8 14:00:28 2008].247322 msHTTPExecuteRequests(): HTTP request error. CURLOPT_PROXYAUTH 
not supported. Requires Curl 7.10.7 and up. *_proxy_auth_type setting ignored.

Leaving ticket open for documentation as per comment:23

[dmorissette]

Note the Curl 7.10.6 patch is in r7790

[jmckenna]

- added proxy metadata to WMS-clients howto doc

[mko]

CURLOPT_PROXYAUTH is not correctly set using libcurl 7.19.7 and MapServer 5.2.1 or 5.4.2 on Linux x86_64.

configure:11858: checking for curl-config
configure:11884: checking for curl-config
configure:11902: found /usr/local/bin/curl-config
configure:11915: result: /usr/local/bin/curl-config
configure:11934: result:         found libcurl version 7.19.7
configure:11945: result:         OGC WMS Client Connections enabled (-DUSE_WMS_LYR).
configure:11951: result:         OGC WFS Client Connections enabled (-DUSE_WFS_LYR).

But mapserv binary produces

CURLOPT_PROXYAUTH not supported. Requires Curl 7.10.7 and up. *_proxy_auth_type setting ignored.

error. Setting CURLOPT_PROXYAUTH manually works fine.

[aboudreault]

The curl proxy auth support has been fixed and committed in r9708.

[dmorissette]

It seems that we forgot to implement proxy support for WFS client layers... ticket #3982 has been created to address this.