Opened 19 years ago

Closed 13 years ago

Last modified 12 years ago

#571 closed defect (fixed)

[WMS/WFS] Ability to specify a proxy for remote connections

Reported by: dmorissette Owned by: aboudreault
Priority: high Milestone: 5.2 release
Component: Documentation - MapServer Version: 4.1
Severity: normal Keywords:
Cc: mapserver@…, bartvde@…, franck.villoin@…, tjmitchell@…, micheal.arnold@…, pierrick.brihaye@…, jcampanello@…, mko

Description (last modified by dmorissette)

We need the ability to specify a proxy when connecting to remote WMS/WFS servers.

This could possibly be added as layer metadata wms_proxy_*, but I wonder if we should instead create global configuration parameters inside the WEB object?

I never really worked in environments with proxies, so I wonder if it is safe to assume that when someone uses a proxy, then they use it for *all* their http connections and not only to get to specific hosts? i.e. is it a good idea to have a single global proxy setting in the web object in the mapfile or should this be local at the layer level?

Attachments (2)

mapserv-571-curl_proxy+auth.patch (13.2 KB ) - added by project10 15 years ago.
Initial implementation
mapserver-curlversion.patch (1.5 KB ) - added by dionw 15 years ago.
allow build against curl < 7.10.7 (e.g. RHEL3)

Download all attachments as: .zip

Change History (38)

comment:1 by fwarmerdam, 19 years ago

I agree that this should be map wide, rather than layer specific.  

I am not sure it belongs in the WEB section though, since it relates to 
how remote requests are made, now how the local configuraton works.  

There are also a number of other global configuration options I would like
to add in the not-to-distant-future, including .map file driven overrides for
GDAL_DATA, and PROJ_LIB.   I'm not sure that these are related though. 

comment:2 by bartvde@…, 19 years ago

The production environments I have been in would only need to have a global 
proxy setting in the MAP file, not a per layer proxy setting.

comment:3 by mapserver@…, 19 years ago

Cc: mapserver@… added

comment:4 by dmorissette, 19 years ago

Milestone: 4.2 release

comment:5 by franck.villoin@…, 19 years ago

Cc: franck.villoin@… added

comment:6 by franck.villoin@…, 19 years ago

Attention, we can imagine use Services WMS without proxy for certain layers in 
the Intranet, and others services WMS/WFS out of intranet use the proxy.
But I use only one the proxy in my production environment. 

comment:7 by dmorissette, 19 years ago

Franck, what would happen in your case if you tried to access a WMS server from
the intranet via the proxy?  Would things still work?

comment:8 by franck.villoin@…, 19 years ago

No, it does not function. 
If I pass by my proxy, the addresses of my Intranet are not solved.
By fixing the proxy in a permanent way in my navigator,
I do not see any more the sites of my Intranet.

Is it possible to define a global proxy parameter, and to make it active by 
layer if necessary?


comment:9 by dmorissette, 19 years ago

Milestone: 4.2 release4.4 release
Pushing to 4.4 release

comment:10 by tjmitchell@…, 19 years ago

Cc: tjmitchell@… added

comment:11 by tjmitchell@…, 19 years ago

I also need this functionality due to some relocation of our servers.  Our 
proxy requires our username/password too!  Ugh.

Curl docs say:

"NOTE2: libcurl respects the environment variables http_proxy, ftp_proxy, 
all_proxy etc, if any of those is set."

Can this be used somehow as a quick workaround?


comment:12 by dmorissette, 19 years ago

The libcurl tutorial talks about those environment variables as well. They must
be lowercase, and the value is in the format
"[protocol://][user:password@]machine[:port]". They also note that the
"protocol://" part is simply ignored if present. (Proxies are always http).

Tyler, I think the best way to go is for you to try this and let us know if that

comment:13 by micheal.arnold@…, 18 years ago

Cc: micheal.arnold@… added

comment:14 by dmorissette, 18 years ago

Milestone: 4.4 releaseFUTURE
Not in 4.4. Setting FUTURE target milsetone.

comment:15 by pierrick.brihaye@…, 18 years ago

Cc: pierrick.brihaye@… added

comment:16 by bartvde@…, 18 years ago

I checked this by using SetEnv http_proxy in the Apache conf and all works fine!
There is also an option in curl to exclude certain domains if needed.

For my use this would be enough. I can't think of any situation in which you
want per layer proxy url's.

comment:17 by jcampanello@…, 17 years ago

Cc: jcampanello@… added

comment:18 by elromero@…, 17 years ago

op_sys: LinuxWindows XP

I'm a new MapServer user and I have a problem. I want to call a layer from a 
remote WMS but I can't because I have a problem whith the proxy.
I have inserted the next line in httpd.conf file:
SetEnv http_proxy "http://isas.vivienda.local:8080"

And I have inserted in my map file:
    NAME         modis_jpl
    TYPE         RASTER
    OFFSITE      0 0 0
    STATUS       OFF
    DEBUG        ON
      "wms_srs" "EPSG:4326"
      "wms_name" "modis"
      "wms_server_version" "1.1.1"
      "wms_format" "image/jpeg"      
      "wms_proxy_*" "isas.vivienda.local:8080"

  END # Modis WMS image ends here

But I only get a temporary file with an autentification proxy problem.

What am I doing bad? What more do I need?

Thak you,


comment:19 by dmorissette, 17 years ago

Emilio, this is a bug report, it is not the place to ask for support, I would
suggest that you send your question to the mapserver-users list. If it turns out
that better proxy support would solve your problem then the conclusions of the
list discussion can be added in support to this bug later on.
(Hint: also include a copy of the proxy error messages in your post to the list.)

comment:20 by dmorissette, 15 years ago

Description: modified (diff)
Milestone: FUTURE5.2 release

by project10, 15 years ago

Initial implementation

comment:21 by project10, 15 years ago

I've attached a patch which provides an initial implementation of the ability to specify proxying information, as well as HTTP authentication information for WMS layers. The implementation uses layer- and web-level metadata, as discussed on #mapserver.

The patch adds the following new metadata directives:

  'wms_proxy_host' ''
  • can be specified as a hostname, dotted-quad, with optional port component (e.g. '')
      'wms_proxy_port' '8080'
  • if the port is not given above, can be specified with this metadata
      'wms_proxy_type' 'http'
  • supported types: 'http', 'socks5' (case insensitive)
      'wms_proxy_auth_type' 'ntlm'
  • supported types (case insensitive):
    • basic
    • digest
    • ntlm
    • any (the underlying http library picks the best among the options supported by the remote server)
    • anysafe (the underlying http library picks only safe methods among the options supported by the remote server)
        'wms_proxy_username' 'foo'
        'wms_proxy_password' '{FF88CFDAAE1A5E33}'
  • msEncrypt-style string supported; also accepts "naked" strings
      'wms_auth_type' 'ntlm'
  • same supported options as wms_proxy_auth_type
      'wms_auth_username' 'foo1'
      'wms_auth_password' '{FF88CFDAAE1A5E33}'
  • msEncrypt-style string supported; also accepts "naked" strings

comment:22 by sdlime, 15 years ago

Owner: changed from mapserverbugs to sdlime

comment:23 by sdlime, 15 years ago

Component: WMS ClientMapServer Documentation
Status: newassigned

I have applied the patches to the dev trunk. If folks could please test and report back I'd appreciate it. I'm moving this to a documentation bug at this point.


comment:24 by dmorissette, 15 years ago

I just committed r7562 with fixes to the r7560 patch to make it compile cleanly, and added MS_ prefix to the proxy/auth type constants to avoid possible symbol collisions in the future (#571).

I don't have a proxy to test with, so I just made sure the build worked.

comment:25 by dmorissette, 15 years ago

Doh! In the comment above I meant r7652 ... which fixes r7650 ... sorry!

And for those who might notice the new maplexer.c as part of the same commit, it was due to get into SVN separately, it is related to ticket #2634 (r7633) and got committed at the same time by accident.

in reply to:  24 comment:26 by sdlime, 15 years ago

Replying to dmorissette:

Dan, thanks for the fixes! -Steve

by dionw, 15 years ago

Attachment: mapserver-curlversion.patch added

allow build against curl < 7.10.7 (e.g. RHEL3)

comment:27 by dionw, 15 years ago

Using CURLOPT_PROXYAUTH is only available starting with curl 7.10.7. To allow building against earlier versions (e.g. RHEL3 comes with 7.10.6) I've created the above patch.

Additionally, the patch introduces a warning when using Mapserver build with curl that doesn't support setting the proxy auth type, but nevertheless supplying the wms_proxy_auth_type metadata in the mapfile. Perhaps this should be an error.

comment:28 by dmorissette, 15 years ago

Applied the patch to build against Curl 7.10.6 and older, and also log an error to warn the user but without aborting the execution. The error will look as follows and will be trappable through the error stack, or visible in the DEBUG output:

[Tue Jul  8 14:00:28 2008].247322 msHTTPExecuteRequests(): HTTP request error. CURLOPT_PROXYAUTH 
not supported. Requires Curl 7.10.7 and up. *_proxy_auth_type setting ignored.

Leaving ticket open for documentation as per comment:23

comment:29 by dmorissette, 15 years ago

Note the Curl 7.10.6 patch is in r7790

comment:30 by jmckenna, 15 years ago

Owner: changed from sdlime to jmckenna
Status: assignednew

comment:31 by jmckenna, 15 years ago

Status: newassigned

comment:32 by jmckenna, 15 years ago

Resolution: fixed
Status: assignedclosed
  • added proxy metadata to WMS-clients howto doc

comment:33 by mko, 13 years ago

Cc: mko added
Resolution: fixed
Status: closedreopened

CURLOPT_PROXYAUTH is not correctly set using libcurl 7.19.7 and MapServer 5.2.1 or 5.4.2 on Linux x86_64.

configure:11858: checking for curl-config
configure:11884: checking for curl-config
configure:11902: found /usr/local/bin/curl-config
configure:11915: result: /usr/local/bin/curl-config
configure:11934: result:         found libcurl version 7.19.7
configure:11945: result:         OGC WMS Client Connections enabled (-DUSE_WMS_LYR).
configure:11951: result:         OGC WFS Client Connections enabled (-DUSE_WFS_LYR).

But mapserv binary produces

CURLOPT_PROXYAUTH not supported. Requires Curl 7.10.7 and up. *_proxy_auth_type setting ignored.

error. Setting CURLOPT_PROXYAUTH manually works fine.

comment:34 by aboudreault, 13 years ago

Owner: changed from jmckenna to aboudreault
Status: reopenednew

comment:35 by aboudreault, 13 years ago

Resolution: fixed
Status: newclosed

The curl proxy auth support has been fixed and committed in r9708.

comment:36 by dmorissette, 12 years ago

It seems that we forgot to implement proxy support for WFS client layers... ticket #3982 has been created to address this.

Note: See TracTickets for help on using tickets.