Opened 5 weeks ago

Last modified 4 weeks ago

#2484 new task

(linkcheck) Use a different User-Agent: on some urls

Reported by: cvvergara Owned by: osgeolive@…
Priority: normal Milestone:
Component: Documentation Keywords:
Cc:

Description

Some url are "broken" maybe because of the User-Agent. Example error: 

broken    https://opensource.org - 403 Client Error: Forbidden for url: https://opensource.org/

The User-Agent might need to be changed to get a response from the url.

Information about how to handle User-Agent:

​https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-linkcheck_request_headers

About User-Agent:

​https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent

Change History (2)

comment:1 by darkblueb, 4 weeks ago

on alpha92

current docs package : osgeolive-docs/noble 17.0~alpha1-0~202410212258~ubuntu24.04.1 all [upgradable from: 17.0~alpha1-0~202409091351~ubuntu24.04.1]

user-agent string "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"

command-line user@osgeolive:~$ linkchecker -v --user-agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0" --check-extern http://localhost

17866 links in 19158 URLs checked. 2292 warnings found. 332 errors found.

Last edited 4 weeks ago by darkblueb (previous) (diff)

comment:2 by darkblueb, 4 weeks ago

lychee 

   58793 Total (in 1m 46s) ✅ 47127 OK 🚫 3208 Errors 👻 73 Excluded ⏳ 56 Timeouts
Note: See TracTickets for help on using tickets.