Google Storage 签名 URL 过期导致 400 错误且没有 CORS 标头

Question

长话短说：对过期签名PUTURL 响应的请求不包含成功上传到未过期签名PUTURL 所包含的 CORS 标头。

我们运营一项允许用户上传视频的服务。我们的后端 API 生成一个签名的 Google 存储 URL，基于浏览器的客户端应用程序可以将块上传到该 URL。

不幸的是，在较慢的连接上，这些 URL 可能在使用之前就过期了。当我们使用 Azure Blob 存储或 Amazon S3 作为存储介质时，我们的重试机制将检测到失败 (403)，为该特定块请求新的 URL，然后继续上传。

在 Google Storage 上，这不起作用，原因很简单：虽然OPTIONS成功返回，并且成功的PUT请求包含access-control-allow-origin标头，但失败的上传（收到 400 状态代码的上传）却没有，并且我们在控制台中记录了以下错误：

访问 XMLHttpRequest ' https://storage.googleapis.com/ourbucket/uploads%2F20190308%2F5c822a942d1bd1000183a4a6%2Flawbarnd.mp4_c6b701ce-1687-4fb5-a453-61875c1b6d9a__000007?GoogleAccessId=user @ project.iam.gserviceaccount.com&Expires=1552034512&Signature=L6pvUQX5UEa7GESO %2Bj12yR8%2FXln3tz1SDUA%2Bkf1NNx9eTvmUxTdgROYo30p4s%2FGGhXYwr%2BUdgnDuZ66pjX7YS0N5PO5BIr6LULtpR6i2xNC8Y2sKmpv5QF66FHqSBWK0YoLc%2B21MnJMPRgUBSXMcoyWJCJ%2FAapVgRe9QH%2BQt86agf6h0yEmHv48qgVJpzRH%2FbiNJKD7oiOyJc%2Fcon2y2hqsCo6x8buZVuPzTZg6ddHqmqKkscjABoT7bq1%2Bz7Sqkq3Vul%2B5XQfw3CvoNjELpuqVQA%2F0v0RXE86JkOnXf2kQKKlL%2Fq9AwidsEMF05n1LlBVRKSdv8qNKTCVFwBOU%2BMg%3D%3D ' from origin ' https://www.example.com ' has been被 CORS 策略阻止：请求的资源上不存在“Access-Control-Allow-Origin”标头。

我们的桶CORS配置如下：

[
    {
        "maxAgeSeconds": 3600,
        "method": ["*"],
        "origin": ["https://www.example.com"],
        "responseHeader": ["*"]
    }
]

我们的客户端上传代码：

this.bytesUploaded = 0;
this.xhr = new XMLHttpRequest();
this.xhr.open(method, url, true);

let keys = _.keys(headers);
if (keys !== null && keys !== undefined && keys.length > 0) {
    for (let i = 0; i < keys.length; ++i) {
        this.xhr.setRequestHeader(keys[i], headers[keys[i]]);
    }
}

this.xhr.upload.addEventListener('progress', this.onProgress);
this.xhr.addEventListener('load', this.onLoad);
this.xhr.addEventListener('error', this.onError);
this.xhr.addEventListener('abort', this.onAbort);

this.xhr.setRequestHeader('Content-Type', ' '); // we unset this because it interferes with signed URLs. This works fine.

this.xhr.send(this.data);

成功请求的响应标头（通过 Fiddler）：

HTTP/1.1 200 OK
X-GUploader-UploadID: AEnB2UrZdsd-DAl0VdYOtKGVD_4AJLf6qeukybq0jBSv5HI5M4fRTqFVnoxko5LJBMttKYz8ExXG1c3BeASH4IuO8iKfCBb-iw
ETag: "87a742c72cc29950f03e5dd86dc95cf4"
x-goog-generation: 1552036072503518
x-goog-metageneration: 1
x-goog-hash: crc32c=u2rTqA==
x-goog-hash: md5=h6dCxyzCmVDwPl3Ybclc9A==
x-goog-stored-content-length: 239170
x-goog-stored-content-encoding: identity
Access-Control-Allow-Origin: https://www.example.com
Access-Control-Expose-Headers: *, Content-Length, Content-Type, Date, Server, Transfer-Encoding, X-GUploader-UploadID, X-Google-Trace
Vary: Origin
Content-Length: 0
Date: Fri, 08 Mar 2019 09:07:52 GMT
Server: UploadServer
Content-Type: text/html; charset=UTF-8
Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"

失败请求的响应（通过 Fiddler）：

HTTP/1.1 400 Bad Request
X-GUploader-UploadID: AEnB2UpjxOthcO6AZgBgw_P8Msw1zeZFkEqMhEWF5pV9jPORajlBnizndw48WSBtW_Ft9G7NOHu_HWxjgywpG7dqhZ0QUz8znA
Content-Type: application/xml; charset=UTF-8
Content-Length: 202
Date: Fri, 08 Mar 2019 09:01:39 GMT
Server: UploadServer
Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"

<?xml version='1.0' encoding='UTF-8'?><Error><Code>ExpiredToken</Code><Message>The provided token has expired.</Message><Details>Request signature expired at: 2019-03-08T09:01:24+00:00</Details></Error>

存储桶上是否缺少配置选项？有没有一种方法可以在发生故障时忽略 CORS 以提取失败的状态代码？目前我们只收到 -1，这没有帮助。

编辑回答 Yasser Karout 的问题：

浏览器为了完成 XHR PUT 请求，首先进行飞行前OPTIONS调用：

OPTIONS https://storage.googleapis.com/example-com-media/uploads%2F20190404%2F5ca556b3d72f640001981487%2Fu1equspb.mp4_2c69f05f-0cb7-4c08-bd20-3858b06f6d51__000005?GoogleAccessId=example-com-media@stalwart-kite-714.iam.gserviceaccount.com&Expires=1554339568&Signature=IWjzT0D3Vxzw96JSTwqclhlJWZ%2B%2FBHYviL9SPnZCT3c5P2%2FSqJaq0Grxc%2BpDNLQ2DABH7LdnINR1ZJWF5TMsHoVyWwcwF5OnOqJiKUaGldKos0XFqwXMWo4c%2F7RN1fnKqBkfeSoQXccqwIxr19fh6NYojc09wDwAggcqmBYPmLv7g%2Bui%2FtkEyRTqs4%2Fw4Csl5kmXcOJliX9EWlOmsaJKlFXOmeQEM1IePtBBf4hjJJ%2FnKeRjfdjdmz1d%2BZ1F2LP6qGHCe5ay%2FSn7%2Fw23GfAaWZHFlcevLxgNuu0dpRW4yN6dTjckpgRonXYupGizMDzkQ7K6d1rKEl5bSpXBROMp7Q%3D%3D HTTP/1.1
Host: storage.googleapis.com
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Access-Control-Request-Method: PUT
Origin: https://www.example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36
Access-Control-Request-Headers: content-type
Accept: */*
X-Client-Data: CJW2yQEIpLbJAQjEtskBCKmdygEIqKPKAQi8pMoBCLGnygEI4qjKAQjxqcoB
Referer: https://www.example.com/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,ja;q=0.8

对此的响应200 OK确实包含CORS 标头：

HTTP/1.1 200 OK
X-GUploader-UploadID: AEnB2UqxbjqsngYYKdvLmrHj21htyUusQkR2W3tge38fMd30TehyRy7wDDmq6U9a7oYIL1OCGJP9hw3uXNVFH8_qbIR-Skhpag
Access-Control-Allow-Origin: https://www.example.com
Access-Control-Max-Age: 3600
Access-Control-Allow-Methods: PUT
Access-Control-Allow-Headers: content-type
Vary: Origin
Date: Thu, 04 Apr 2019 01:00:39 GMT
Expires: Thu, 04 Apr 2019 01:00:39 GMT
Cache-Control: private, max-age=0
Content-Length: 0
Server: UploadServer
Content-Type: text/html; charset=UTF-8
Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"

因为这是可以的，浏览器随后会发出请求PUT：

PUT https://storage.googleapis.com/example-com-media/uploads%2F20190404%2F5ca556b3d72f640001981487%2Fu1equspb.mp4_2c69f05f-0cb7-4c08-bd20-3858b06f6d51__000005?GoogleAccessId=example-com-media@stalwart-kite-714.iam.gserviceaccount.com&Expires=1554339568&Signature=IWjzT0D3Vxzw96JSTwqclhlJWZ%2B%2FBHYviL9SPnZCT3c5P2%2FSqJaq0Grxc%2BpDNLQ2DABH7LdnINR1ZJWF5TMsHoVyWwcwF5OnOqJiKUaGldKos0XFqwXMWo4c%2F7RN1fnKqBkfeSoQXccqwIxr19fh6NYojc09wDwAggcqmBYPmLv7g%2Bui%2FtkEyRTqs4%2Fw4Csl5kmXcOJliX9EWlOmsaJKlFXOmeQEM1IePtBBf4hjJJ%2FnKeRjfdjdmz1d%2BZ1F2LP6qGHCe5ay%2FSn7%2Fw23GfAaWZHFlcevLxgNuu0dpRW4yN6dTjckpgRonXYupGizMDzkQ7K6d1rKEl5bSpXBROMp7Q%3D%3D HTTP/1.1
Host: storage.googleapis.com
Connection: keep-alive
Content-Length: 1332887
Pragma: no-cache
Cache-Control: no-cache
Origin: https://www.example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36
Content-Type: 
Accept: */*
X-Client-Data: CJW2yQEIpLbJAQjEtskBCKmdygEIqKPKAQi8pMoBCLGnygEI4qjKAQjxqcoB
Referer: https://www.example.com/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,ja;q=0.8

在 Telerik 的 Fiddler 中查看 HTTP 请求我可以看到以下响应：

HTTP/1.1 400 Bad Request
X-GUploader-UploadID: AEnB2UqdCe0t1tcfu_vgw7xirkbY6ACX_rZRac4UuufCU5vLufAsFQIQ06uNuE7zzCg7u8OXZN0aEu5ygD7TAJdqv4kVkBDf0w
Content-Type: application/xml; charset=UTF-8
Content-Length: 202
Date: Thu, 04 Apr 2019 01:00:39 GMT
Server: UploadServer
Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"

<?xml version='1.0' encoding='UTF-8'?><Error><Code>ExpiredToken</Code><Message>The provided token has expired.</Message><Details>Request signature expired at: 2019-04-04T00:59:28+00:00</Details></Error>

因此，回答 Yasser 的问题：是的，400返回了状态代码，但由于不存在 CORS 标头，浏览器永远不会将该响应提供给调用 JavaScript 代码，因此不可能知道请求失败的原因。可以肯定地说，请求失败了。

Answer 1

感谢您的澄清。进一步研究后发现，这似乎是当前过期签名 URL 的预期行为。标头“Access-Control-Allow-Origin”标头不存在导致错误。

对此有一个开放的功能请求，它也链接到存储团队的内部功能请求。我也会在内部提供这个堆栈帖子作为额外的用例。