Opened 7 months ago

Closed 7 months ago

Last modified 7 months ago

#7116 closed defect (fixed)

vsis3: AWS HTTP 307 redirect responses fails to authenticate

Reported by: tveastman Owned by: warmerdam
Priority: normal Milestone:
Component: default Version: 2.2.2
Severity: normal Keywords: VSIS3 S3 AWS 307 redirect signature
Cc: robert.coup@…

Description

We've found that many buckets fail to be listed using /vsis3/

Amazon AWS will sometimes return a 'HTTP/1.1 307 Temporary Redirect', requiring the client to make the same request to a different endpoint. This is common with newly created buckets.

For example, a call to list the contents of 'bucket-name.s3.amazonaws.com' can return a 307 redirect to 'kx-test-s3-scan-import.s3-ap-southeast-2.amazonaws.com'.

The /vsis3/ handler follows this redirect, but it does not seem to update the 'Authorization' header for the request.

Because the 'Host' header has changed, the Authorization header is no longer valid, and the request fails with '403 Forbidden'.

It seems that what's required is for the Authorization: header to be rebuilt before making the follow-up (redirected) request.

AWS docs on Temporary Request Redirection are at: https://docs.aws.amazon.com/AmazonS3/latest/dev/Redirects.html

I haven't found AWS docs yet that *explicitly* state the Authorization header needs to be recalculated, but I note that 'host' is in the list of 'SignedHeaders?', and the Host header *does* change after redirect, while the signature does not.

Change History (6)

comment:1 Changed 7 months ago by tveastman

Here's a transcript of the two curl requests made after an attempt to read the directory, I have redacted my AWS_ACCESS_KEY_ID

The smoking gun, I think, is that the Host header changes between the two requests, the Authorization header specifies SignedHeaders=host, and yet the signature has not been updated.

In [33]: gdal.ReadDir('/vsis3/kx-test-s3-scan-import/')

> GET /?delimiter=/ HTTP/1.1
Host: kx-test-s3-scan-import.s3.amazonaws.com
Accept: */*
x-amz-date: 20171031T013051Z
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256 Credential=REDACTED/20171031/ap-southeast-2/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=973a3c2db94867c18b19f808290608fc7f54b0b359d33a50980b8d58b0c3ed73

< HTTP/1.1 307 Temporary Redirect
< x-amz-bucket-region: ap-southeast-2
< x-amz-request-id: 5A712999630381D5
< x-amz-id-2: JfErqIqhkOEhXAaSxMylOKGEhUhLhsM+r8EtI3tkf1G3DuFyWRPC4FXxuIUmzc5GmN4CzPluZh8=
< Location: https://kx-test-s3-scan-import.s3-ap-southeast-2.amazonaws.com/?delimiter=/
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Tue, 31 Oct 2017 01:30:51 GMT
* Server AmazonS3 is not blacklisted
< Server: AmazonS3


> GET /?delimiter=/ HTTP/1.1
Host: kx-test-s3-scan-import.s3-ap-southeast-2.amazonaws.com
Accept: */*
x-amz-date: 20171031T013051Z
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256 Credential=REDACTED/20171031/ap-southeast-2/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=973a3c2db94867c18b19f808290608fc7f54b0b359d33a50980b8d58b0c3ed73

< HTTP/1.1 403 Forbidden
< x-amz-bucket-region: ap-southeast-2
< x-amz-request-id: 8E0DA2DD9CF89026
< x-amz-id-2: 8YfrNVucdfN1+WDV6ZHkF0Gf28MxodR5vR6SpMzJlTJWiKfUF7uKMKUHoEzDfKryBQme1xK5z5g=
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Tue, 31 Oct 2017 01:30:51 GMT
* Server AmazonS3 is not blacklisted
< Server: AmazonS3
<
* Connection #12 to host kx-test-s3-scan-import.s3-ap-southeast-2.amazonaws.com left intact
Last edited 7 months ago by tveastman (previous) (diff)

comment:2 Changed 7 months ago by Even Rouault

Resolution: fixed
Status: newclosed

In 40591:

/vsis3/: properly handle 307 TemporaryRedirection? (fixes #7116)

comment:3 Changed 7 months ago by Even Rouault

Would you mind retrying with latest trunk ? I didn't try to reproduce with the "real" S3 servers, but tested against the stub used for autotest suite.

comment:4 Changed 7 months ago by tveastman

I can't test the patch yet, but when I can I'll update this ticket with my results.

comment:5 Changed 7 months ago by tveastman

I've tested a 2.3 build, and the signatures are now updated on redirect, and the call succeeds.

However, there's a 400 BAD REQUEST response in the request logs, I wanted to have you look and check if that's expected behaviour or not.

Here's an example call to a test bucket I created, I've redacted my AWS_ACCESS_KEY_ID. You'll see a couple calls, including a 400 response, and a successful result.

In other words, looks like the bug is fixed, but is there something else going wrong?

> GET /?delimiter=/ HTTP/1.1
Host: an-irish-bucket.s3.amazonaws.com
Accept: */*
x-amz-date: 20171105T214501Z
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256 Credential=AKIAREDACTED/20171105/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=6fd339cbe15000c65447219bbd3894b086a5de8f7acd75ab0b782b501160386a

< HTTP/1.1 307 Temporary Redirect
< x-amz-bucket-region: eu-west-1
< x-amz-request-id: B44BE218EBCA093A
< x-amz-id-2: BXi5XGEqC/CeTm/q123XmV5aye3E1/pORy6WedOJlEBKYvtl0MuYMOeXB4yUIP43pSn2UdaFFvA=
< Location: https://an-irish-bucket.s3-eu-west-1.amazonaws.com/?delimiter=/
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Sun, 05 Nov 2017 21:45:02 GMT
< Server: AmazonS3

> GET /?delimiter=/ HTTP/1.1
Host: an-irish-bucket.s3-eu-west-1.amazonaws.com
Accept: */*
x-amz-date: 20171105T214502Z
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256 Credential=AKIAREDACTED/20171105/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=81a0717fb17dd5db8ac422c774ca8b3a94b5cd249621be822d1ff746de0896b1

< HTTP/1.1 400 Bad Request
< x-amz-bucket-region: eu-west-1
< x-amz-request-id: 1E8E55536F81A634
< x-amz-id-2: c475qXUrpdJfH0h997Es3x9GIeoqSEMlkSeowMIvwEBAFBLnfGalQ07ZlkIirbyoA8M9+UAbYFo=
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Sun, 05 Nov 2017 21:45:04 GMT
< Connection: close
< Server: AmazonS3


> GET /?delimiter=/ HTTP/1.1
Host: an-irish-bucket.s3-eu-west-1.amazonaws.com
Accept: */*
x-amz-date: 20171105T214504Z
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256 Credential=AKIAREDACTED/20171105/eu-west-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=f7af380f40e5de098880cbab5e40a599c1075f74c4d65253f2c746eebf191c78

< HTTP/1.1 200 OK
< x-amz-id-2: 204lMzM7t5dEOEOq+NMkuSlPnxOnV2nvuTlsWJ+eDouEeTM0h/QbjdGrBEq/Z06FmY18wzeArII=
< x-amz-request-id: 5CA4CA211D857B47
< Date: Sun, 05 Nov 2017 21:45:06 GMT
< x-amz-bucket-region: eu-west-1
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Server: AmazonS3
['a-directory']

comment:6 Changed 7 months ago by Even Rouault

This is more or less expected. The code doesn't handle the "x-amz-bucket-region: eu-west-1" response header that indicates the change of region, so it first retry with the currrent/default region (us-east-1) and the 400 response will give it the new region, so there's some potential for optimization there

Note: See TracTickets for help on using tickets.