使用查询参数将 AWS 预签名 URL 改革为基于标头的授权

问题描述 投票：0回答：1

我有一个用于获取请求的预签名 URL，例如：

https://<my-bucket>.s3.amazonaws.com/<path/to/file.dat>?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=<credential-string>%2F20241107%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241107T202532Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-string>

我想使用 

httr2
缓存，因此我想使用基于标头的授权重新构造此 URL，以便缓存不会仅仅因为日期和凭据更改而失效。但是，我无法使基于标头的请求正常工作。这是我正在尝试的，基于此文档：

aws_authorization = "AWS4-HMAC-SHA256Credential=<credential-string>%2F20241107%2Fus-east-1%2Fs3%2Faws4_request,SignedHeaders=host;x-amz-date=20241107T202532Z;x-amz-expires=900,Signature=<signature-string>"

为了清楚起见，这里再次显示带有换行符的授权标头：

aws_authorization = "AWS4-HMAC-SHA256
Credential=<credential-string>%2F20241107%2Fus-east-1%2Fs3%2Faws4_request,
SignedHeaders=host;x-amz-date=20241107T202532Z;x-amz-expires=900,
Signature=<signature-string>"

然后我将请求重构为

req = request("https://<my-bucket>.s3.amazonaws.com/<path/to/file.dat>") |>
  req_headers(Authorization = aws_authorization)

req
#> <httr2_request>
#> GET
#> https://<my-bucket>.s3.amazonaws.com/<path/to/file.dat>
#> Headers:
#> • Authorization:
#> 'AWS4-HMAC-SHA256Credential=<credential-string>/20241107/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-date=20241107T202532Z;x-amz-expires=900,Signature=<signature-string>'
#> Body: empty

但是当我执行请求时，我收到 HTTP 400 Bad Request 错误：

req_perform(req)
#> HTTP 400 Bad Request.

我在构造请求时哪里出错了？

编辑

根据评论修改：

  1. 从 SignedHeaders 中删除 
    x-amz-expires
    。
  2. 从 SignedHeaders 中删除 
    =
    ，并定义引用的标头。

所以现在我的授权字符串是

AWS4-HMAC-SHA256Credential=<credential-string>%2Fus-east-1%2Fs3%2Faws4_request,SignedHeaders=host;x-amz-date,Signature=<signature-string>

我已经添加了标题

X-Amz-Date = 20241107T202532Z
X-Amz-Expires = 900

产生请求

#> <httr2_request>
#> GET
#> <my-bucket>.s3.amazonaws.com/<path/to/file.dat>
#> Headers:
#> • Authorization:
#> 'AWS4-HMAC-SHA256Credential=<credential-string>%2F20241107%2Fus-east-1%2Fs3%2Faws4_request,SignedHeaders=host;x-amz-date,Signature=<signature-string>'
#> • X-Amz-Date: '20241107T220527Z'
#> • X-Amz-Expires: '900'
#> Body: empty
r amazon-s3 httr2
1个回答
0
投票

以下是使用以下方法完成的：

  • R版本4.4.2（2024-10-31）
  • httr2 软件包版本 1.0.6.9000
  • macOS 索诺玛 14.6.1

httr2 1.0.6包含一个函数req_auth_aws_v4()，它接近你想要的，但我无法让它与AWS s3一起使用。但是，如果您想创建自己的签名算法，req_auth_aws_v4() 代码很有用。

AWS 在对不成功的签名请求的响应中提供有用的调试信息。如果您看到 400 Bad Request 响应，我会尝试以下操作来阻止 httr2 抛出错误，以便您可以看到响应内容：

req = request("https://<my-bucket>.s3.amazonaws.com/<path/to/file.dat>") |> req_headers(Authorization = aws_authorization)
    
req <- req_verbose(
  req,
  header_req = TRUE,
  header_resp = TRUE,
  body_req = TRUE,
  body_resp = TRUE,
  info = TRUE,
  redact_headers = FALSE
)

tryCatch(
  resp <- req |> req_perform(),
  httr2_http_400 = function(cnd) {
   last_response() |> resp_raw()
  }

我使用 Python requests 编写（借用）了以下代码，该代码使用标头身份验证成功返回了 AWS S3 对象：

import sys, os, base64, datetime, hashlib, hmac
import requests
method = 'GET'
service = 's3'
host = '<my-bucket>.s3.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://<my-bucket>.s3.amazonaws.com'
request_parameters = ''

def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

access_key = "AABBCCDDEEFF11223344"
secret_key = "SomeBigLongSecretKeyToUseForAWSDontShare"

if access_key is None or secret_key is None:
    print('No access key is available.')
    sys.exit()

t = datetime.datetime()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope

canonical_uri = '/<path/to/file.dat>'
canonical_querystring = request_parameters
canonical_headers = 'host:' + host + '\n' + 'x-amz-content-sha256:UNSIGNED-PAYLOAD' + '\n' + 'x-amz-date:' + amzdate + '\n'
signed_headers = 'host;x-amz-content-sha256;x-amz-date'
payload_hash = 'UNSIGNED-PAYLOAD'
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'

print(canonical_request)

string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' +  hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
signing_key = getSignatureKey(secret_key, datestamp, region, service)
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()

print(signature)

authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature
headers = {'x-amz-date':amzdate, 'x-amz-content-sha256': 'UNSIGNED-PAYLOAD', 'Authorization':authorization_header}
request_url = endpoint + canonical_uri

print(authorization_header)
print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + request_url)

r = requests.get(request_url, headers=headers)

print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)

这里要注意的重要事项是它发送以下标头：

  • x-amz-日期
  • x-amz-内容-sha256
  • 授权

并且 Authorization 标头格式为：

AWS4-HMAC-SHA256 Credential=AKIASMI5W6NBQAMBZHR5/20241110/ap-southeast-2/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=2ad745df5f4680d42800865d775ca045651eb42819e18592b99c43c3c00b949f

注意

算法
和凭证之间的空格' '分隔符：

AWS4-HMAC-SHA256 Credential

httr2 req_auth_aws_v4() 函数似乎生成一个授权标头，该标头在算法和凭证之间有 ','

，这可能是问题所在：

AWS4-HMAC-SHA256,Credential 
如果您有有效的签名，则以下标头似乎可以使用 
httr2:

library(httr2)

aws_date = "20241110"
aws_time = "170747"
sig = "64aa96f4dceaa3b782245ba9a070621190e1feab93c349bc6a5a83a8ce8ee4e3"

# Tested with AWS S3
#
# The Authorization header for AWSV4 appears to require a space " " after the Algorithm AWS4-HMAC-SHA256
#

aws_authorization = paste("AWS4-HMAC-SHA256 Credential=AABBCCDDEEFF11223344/", aws_date, "/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=", sig, sep = "")

req <- request("https://<my-bucket>.s3.amazonaws.com")
req <- req |> req_url_path(path = "<path/to/file.dat")
req <- req |> req_headers("Authorization" = aws_authorization, "host" = "<my-bucket>.s3.amazonaws.com", "x-amz-content-sha256" = "UNSIGNED-PAYLOAD", "x-amz-date" = paste(aws_date,"T",aws_time,"Z", sep = ""))

req <- req_verbose(
  req,
  header_req = TRUE,
  header_resp = TRUE,
  body_req = TRUE,
  body_resp = TRUE,
  info = TRUE,
  redact_headers = FALSE
)

req

tryCatch(
  resp <- req |> req_perform(),
  httr2_http_400 = function(cnd) {
   last_response() |> resp_raw()
  }
)

resp |> resp_body_raw()

希望通过对 
httr2 req_auth_aws_v4() 函数进行一些更改，将来可以将其简化为以下内容。

library(httr2)

req <- request("https:/<my-bucket>.s3.amazonaws.com/<path/to/file.dat>")

req <- req_auth_aws_v4(
  req,
  aws_access_key_id = "AABBCCDDEEFF11223344",
  aws_secret_access_key = "SomeBigLongSecretKeyToUseForAWSDontShare",
  aws_service = "s3",
  aws_region = "us-east-1"
)

resp <- req_perform(req)

resp

    

	

    

    


  

  

    
最新问题


  





  

    © www.soinside.com 2019 - 2024. All rights reserved.