Fighting Bandwidth Accelerators

[warmerdam]

It would be helpful to have a strategy to limit bandwidth accelerators. That is, situations where one IP open many connections, often on the same file - in an apparent attempt to download it quickly. In some cases this seems to be misconfigured and result in repeated downloads, and in other cases it just ties up a lot of server slots, potentially denying other users service.

One suggestion has been to use mod_cband to limit bandwidth, but this doesn't really stop multiple connections at once, and might limit legitimate high bandwidth use. A also, the mod_cband site (​http://cband.linux.pl) does not seem to be available.

Another promising looking option is mod_limitipconn:

​http://dominia.org/djao/limitipconn2.html

[warmerdam]

I'm going to pursue limitipconn2 on download.osgeo.org.

[warmerdam]

I have built and installed mod_limitipconn2 from source.

I changed the download configuration to:

<Directory "/osgeo/download">
  Options FollowSymLinks Indexes
  MaxConnPerIP 3
</Directory>

A brief test indicates that the 4th simultaneous request from a single IP will receive a 503 (service temporarily unavailable) response.

Mission accomplished!

[warmerdam]

I have put the same thing into place on xblade 14-2 (aka www.gdal.org, upload.osgeo.org, grass.osgeo.org, etc) but I attached the MaxConnPerIP 3 to <Directory "/"> in /etc/httpd/conf/httpd.conf which seems to have applied it to all the virtual hosts on the machine.

[neteler]

The settings are too limited. I cannot even pen a single page with several screenshots on it, see for example:

​http://grass.osgeo.org/screenshots/gui.php

This is a showstopper for us... Please give us more bandwidth.

thanks Markus

[neteler]

I have added an exception for GRASS on that server and especially enabled images to not be blocked:

( [root@xblade14-2 conf]# vim /etc/init.d/httpd )

# docs: http://dominia.org/djao/limitipconn2-README
# grant more to GRASS to not block screenshot pages:
<Directory "/osgeo/grass/rsync">
  Options FollowSymLinks Indexes
  MaxConnPerIP 12
  # exempting images from the connection limit is often a good
  # idea if your web page has lots of inline images, since these
  # pages often generate a flurry of concurrent image requests
  NoIPLimit images/* screenshots/*
</Directory>

Seems to help.

Markus

[hamish]

Hi,

FWIW I was seeing this even though I have not done anything so rude as to bump up the max connection per ip in my browser. I think that simply "3" is too agressive.

hmm. In Debian/stable's Firefox 2.0 if you go to "about:config" and filter on connection you will see:

network.http.max-connections-per-server             default   int  8
network.http.max-persistent-connections-per-proxy   default   int  4
network.http.max-persistent-connections-per-server  default   int  2

same values in the epiphany browser.

Markus:

Seems to help.

same here.

thanks, Hamish

[TemptorSent]

Modern browsers and download agents utilize far more connections than they did when this bug was opened. If connection limits are really necessary, they should be set to at the very least 8, but preferably eliminated entirely and serious abuse handled at the network layer. If it's specifically downloads of large files that is causing concern, limits should be imposed only with that specific scope and/or a better means for handling very large files should be identified and implemented.

[cvvergara]

MaxConnPerIP No longer been used in download. Based on the previous comment, I will mark it as invalid. (was a valid issue before, but not anymore)