Ticket #571 (closed defect: fixed)

Opened 5 years ago

Last modified 2 months ago

[WMS/WFS] Ability to specify a proxy for remote connections

Reported by: dmorissette Assigned to: jmckenna
Priority: high Milestone: 5.2 release
Component: MapServer Documentation Version: 4.1
Severity: normal Keywords:
Cc: mapserver@geochem.de, bartvde@osgis.nl, franck.villoin@interieur.gouv.fr, tjmitchell@riverside.bc.ca, micheal.arnold@nscorp.com, pierrick.brihaye@culture.gouv.fr, jcampanello@fibertel.com.ar

Description (Last modified by dmorissette)

We need the ability to specify a proxy when connecting to remote WMS/WFS servers.

This could possibly be added as layer metadata wms_proxy_*, but I wonder if we should instead create global configuration parameters inside the WEB object?

I never really worked in environments with proxies, so I wonder if it is safe to assume that when someone uses a proxy, then they use it for *all* their http connections and not only to get to specific hosts? i.e. is it a good idea to have a single global proxy setting in the web object in the mapfile or should this be local at the layer level?

Attachments

mapserv-571-curl_proxy+auth.patch (13.2 kB) - added by project10 on 04/30/08 20:42:15.
Initial implementation
mapserver-curlversion.patch (1.5 kB) - added by dionw on 07/08/08 05:21:26.
allow build against curl < 7.10.7 (e.g. RHEL3)

Change History

02/19/04 00:15:55 changed by fwarmerdam

I agree that this should be map wide, rather than layer specific.  

I am not sure it belongs in the WEB section though, since it relates to 
how remote requests are made, now how the local configuraton works.  

There are also a number of other global configuration options I would like
to add in the not-to-distant-future, including .map file driven overrides for
GDAL_DATA, and PROJ_LIB.   I'm not sure that these are related though. 

02/19/04 04:57:54 changed by bartvde@osgis.nl

The production environments I have been in would only need to have a global 
proxy setting in the MAP file, not a per layer proxy setting.

02/23/04 11:33:54 changed by mapserver@geochem.de

  • cc set to mapserver@geochem.de.

02/23/04 18:16:05 changed by dmorissette

  • milestone set to 4.2 release.

04/08/04 07:59:09 changed by franck.villoin@interieur.gouv.fr

  • cc set to franck.villoin@interieur.gouv.fr.

04/08/04 08:15:46 changed by franck.villoin@interieur.gouv.fr

Attention, we can imagine use Services WMS without proxy for certain layers in 
the Intranet, and others services WMS/WFS out of intranet use the proxy.
But I use only one the proxy in my production environment. 

04/08/04 11:01:36 changed by dmorissette

Franck, what would happen in your case if you tried to access a WMS server from
the intranet via the proxy?  Would things still work?

04/09/04 05:03:38 changed by franck.villoin@interieur.gouv.fr

No, it does not function. 
If I pass by my proxy, the addresses of my Intranet are not solved.
By fixing the proxy in a permanent way in my navigator,
I do not see any more the sites of my Intranet.

Is it possible to define a global proxy parameter, and to make it active by 
layer if necessary?

Franck

04/14/04 01:26:56 changed by dmorissette

  • milestone changed from 4.2 release to 4.4 release.
Pushing to 4.4 release

08/09/04 16:57:34 changed by tjmitchell@riverside.bc.ca

  • cc set to tjmitchell@riverside.bc.ca.

08/09/04 19:11:42 changed by tjmitchell@riverside.bc.ca

I also need this functionality due to some relocation of our servers.  Our 
proxy requires our username/password too!  Ugh.

Curl docs say:

"NOTE2: libcurl respects the environment variables http_proxy, ftp_proxy, 
all_proxy etc, if any of those is set."

Can this be used somehow as a quick workaround?

Tyler

08/10/04 11:22:03 changed by dmorissette

The libcurl tutorial talks about those environment variables as well. They must
be lowercase, and the value is in the format
"[protocol://][user:password@]machine[:port]". They also note that the
"protocol://" part is simply ignored if present. (Proxies are always http).

Tyler, I think the best way to go is for you to try this and let us know if that
works.

11/22/04 10:44:34 changed by micheal.arnold@nscorp.com

  • cc set to micheal.arnold@nscorp.com.

11/22/04 11:52:45 changed by dmorissette

  • milestone changed from 4.4 release to FUTURE.
Not in 4.4. Setting FUTURE target milsetone.

01/10/05 08:51:52 changed by pierrick.brihaye@culture.gouv.fr

  • cc set to pierrick.brihaye@culture.gouv.fr.

04/05/05 06:58:38 changed by bartvde@osgis.nl

I checked this by using SetEnv http_proxy in the Apache conf and all works fine!
There is also an option in curl to exclude certain domains if needed.

For my use this would be enough. I can't think of any situation in which you
want per layer proxy url's.

12/20/05 16:58:15 changed by jcampanello@fibertel.com.ar

  • cc set to jcampanello@fibertel.com.ar.

07/27/06 04:04:39 changed by elromero@vivienda.es

  • op_sys changed from Linux to Windows XP.
Hello,

I'm a new MapServer user and I have a problem. I want to call a layer from a 
remote WMS but I can't because I have a problem whith the proxy.
I have inserted the next line in httpd.conf file:
SetEnv http_proxy "http://isas.vivienda.local:8080"

And I have inserted in my map file:
LAYER # MODIS WMS map from JPL
    NAME         modis_jpl
    TYPE         RASTER
    OFFSITE      0 0 0
    STATUS       OFF
    CONNECTIONTYPE WMS
    DEBUG        ON
    CONNECTION "http://wms.jpl.nasa.gov/wms.cgi?"
    
    METADATA
      "wms_srs" "EPSG:4326"
      "wms_name" "modis"
      "wms_server_version" "1.1.1"
      "wms_format" "image/jpeg"      
      "wms_proxy_*" "isas.vivienda.local:8080"
    END

    PROJECTION
      "init=epsg:4326"
    END
  END # Modis WMS image ends here

But I only get a temporary file with an autentification proxy problem.

What am I doing bad? What more do I need?

Thak you,

Emilio

07/27/06 06:11:05 changed by dmorissette

Emilio, this is a bug report, it is not the place to ask for support, I would
suggest that you send your question to the mapserver-users list. If it turns out
that better proxy support would solve your problem then the conclusions of the
list discussion can be added in support to this bug later on.
(Hint: also include a copy of the proxy error messages in your post to the list.)

04/30/08 14:33:10 changed by dmorissette

  • description changed.
  • milestone changed from FUTURE to 5.2 release.

04/30/08 20:42:15 changed by project10

  • attachment mapserv-571-curl_proxy+auth.patch added.

Initial implementation

04/30/08 20:50:16 changed by project10

I've attached a patch which provides an initial implementation of the ability to specify proxying information, as well as HTTP authentication information for WMS layers. The implementation uses layer- and web-level metadata, as discussed on #mapserver.

The patch adds the following new metadata directives:

  'wms_proxy_host' '192.168.2.10'
  • can be specified as a hostname, dotted-quad, with optional port component (e.g. '192.168.2.10:8080')
      'wms_proxy_port' '8080'
    
  • if the port is not given above, can be specified with this metadata
      'wms_proxy_type' 'http'
    
  • supported types: 'http', 'socks5' (case insensitive)
      'wms_proxy_auth_type' 'ntlm'
    
  • supported types (case insensitive):
    • basic
    • digest
    • ntlm
    • any (the underlying http library picks the best among the options supported by the remote server)
    • anysafe (the underlying http library picks only safe methods among the options supported by the remote server)
        'wms_proxy_username' 'foo'
        'wms_proxy_password' '{FF88CFDAAE1A5E33}'
      
  • msEncrypt-style string supported; also accepts "naked" strings
      'wms_auth_type' 'ntlm'
    
  • same supported options as wms_proxy_auth_type
      'wms_auth_username' 'foo1'
      'wms_auth_password' '{FF88CFDAAE1A5E33}'
    
  • msEncrypt-style string supported; also accepts "naked" strings

06/01/08 10:57:21 changed by sdlime

  • owner changed from mapserverbugs to sdlime.

06/06/08 10:09:52 changed by sdlime

  • status changed from new to assigned.
  • component changed from WMS Client to MapServer Documentation.

I have applied the patches to the dev trunk. If folks could please test and report back I'd appreciate it. I'm moving this to a documentation bug at this point.

Steve

(follow-up: ↓ 26 ) 06/06/08 15:35:42 changed by dmorissette

I just committed r7562 with fixes to the r7560 patch to make it compile cleanly, and added MS_ prefix to the proxy/auth type constants to avoid possible symbol collisions in the future (#571).

I don't have a proxy to test with, so I just made sure the build worked.

06/06/08 15:41:17 changed by dmorissette

Doh! In the comment above I meant r7652 ... which fixes r7650 ... sorry!

And for those who might notice the new maplexer.c as part of the same commit, it was due to get into SVN separately, it is related to ticket #2634 (r7633) and got committed at the same time by accident.

(in reply to: ↑ 24 ) 06/06/08 16:00:09 changed by sdlime

Replying to dmorissette:

Dan, thanks for the fixes! -Steve

07/08/08 05:21:26 changed by dionw

  • attachment mapserver-curlversion.patch added.

allow build against curl < 7.10.7 (e.g. RHEL3)

07/08/08 05:34:39 changed by dionw

Using CURLOPT_PROXYAUTH is only available starting with curl 7.10.7. To allow building against earlier versions (e.g. RHEL3 comes with 7.10.6) I've created the above patch.

Additionally, the patch introduces a warning when using Mapserver build with curl that doesn't support setting the proxy auth type, but nevertheless supplying the wms_proxy_auth_type metadata in the mapfile. Perhaps this should be an error.

07/08/08 14:02:20 changed by dmorissette

Applied the patch to build against Curl 7.10.6 and older, and also log an error to warn the user but without aborting the execution. The error will look as follows and will be trappable through the error stack, or visible in the DEBUG output:

[Tue Jul  8 14:00:28 2008].247322 msHTTPExecuteRequests(): HTTP request error. CURLOPT_PROXYAUTH 
not supported. Requires Curl 7.10.7 and up. *_proxy_auth_type setting ignored.

Leaving ticket open for documentation as per comment:23

07/08/08 14:03:09 changed by dmorissette

Note the Curl 7.10.6 patch is in r7790

07/15/08 11:58:59 changed by jmckenna

  • status changed from assigned to new.
  • owner changed from sdlime to jmckenna.

07/15/08 11:59:07 changed by jmckenna

  • status changed from new to assigned.

07/15/08 13:38:40 changed by jmckenna

  • status changed from assigned to closed.
  • resolution set to fixed.

- added proxy metadata to WMS-clients howto doc