/vsicurl/: many client-server round-trips with redirection to AWS S3 signed URLs

[Even Rouault]

Some services provide public URLs that are in fact redirections to AWS S3 signed URLs.

There are a few issues with that :

• in most cases those services will return a URL only valid for GET request, but we might try to use it for HEAD to get the file size. So implement a logic to retry from a HEAD to a GET when we detect that we got redirected to a AWS S3 signed URL. Implemented in r33870

• for GET range requests, we will query the public URL, and get redirected each time. This is inefficient and we could directly use the redirection URL. Furthermore for some of the services, range requests on the public URL do not work (probably a shortcoming of the server), so we have to actually query the redirected URL. One tricky point of using the AWS S3 signed URL is the management of the expiration of the URL.

[Even Rouault]

trunk r33876 "/vsicurl/: manage redirection from public URLs to redirected AWS S3 signed urls, with management of the expiration. Enabled by default. Can be disabled if CPL_VSIL_CURL_USE_S3_REDIRECT=NO (#6439)"

[Even Rouault]

r33879 "/vsicurl/: fix regression of r33876 that threw false positive errors about lack of support of range download"