r.in.wms sleep parameter

[frankie]

Some WMS servers have severe limitations about number of tiles downloadable per time units, and ban too much aggressive downloaders. It would be nice having a parameter to add a delay in seconds before downloading each tile. That would be useful for grabbing bots and to be more nice with such servers.

[hamish]

wget has some nice command line options here, you can pass those along from the r.in.wms command parameters. Try --wait=num_seconds or --random-wait I'm not sure if curl does or not.

I assume the .py version does not use wget or curl at all and needs something else?

Hamish

[frankie]

Replying to hamish:

wget has some nice command line options here, you can pass those along from the r.in.wms command parameters. Try --wait=num_seconds or --random-wait I'm not sure if curl does or not.

Good idea, but curl has not a delaying option and I'm not sure if wget has a minimal version to be considered to present those options, so a backend independent implementation is more viable.

[hamish]

new development is in r.in.wms2(.py). bumping there.

[turek]

Some time ago I implemented feature in r56554 which repeats request if server returns 104 error (I came across this error on a server, which sent it if certain number of requests within time limit had been exceeded). If so, request is repeated after 5 seconds then after 30 seconds. Probably the r.in.wms module should repeat requests for another errors returned form a server.

I do not thing that adding new delay parameter into r.in.wms is good idea because this issue should be addressed internally as it is now (too low level for a user).

[martinl]

No activity last 2 years, closing. Feel free to reopen if needed.