如何将AWS S3 URL转换为boto的存储桶名称？

Question

我正在尝试使用boto 访问http://s3.amazonaws.com/commoncrawl/parse-output/segment/存储桶.我无法弄清楚如何将其转换为boto.s3.bucket.Bucket()的名称.

这是我想要的主旨:

s3 = boto.connect_s3()
cc = boto.s3.bucket.Bucket(connection=s3, name='commoncrawl/parse-output/segment')
requester = {'x-amz-request-payer':'requester'}
contents = cc.list(headers=requester)
for i,item in enumerate(contents):
    print item.__repr__()

我得到"boto.exception.S3ResponseError:S3ResponseError:400 Bad Request ...指定的存储桶无效......"

Answer 1

在AWS文件列出四种可能的URL格式S3 -这里的东西我只是把一起提取桶和地区所有的不同的URL格式.

import re

def bucket_name_from_url(url):
    """ Gets bucket name and region from url, matching any of the different formats for S3 urls 
    * http://bucket.s3.amazonaws.com
    * http://bucket.s3-aws-region.amazonaws.com
    * http://s3.amazonaws.com/bucket
    * http://s3-aws-region.amazonaws.com/bucket

    returns bucket name, region
    """       
    match =  re.search('^https?://(.+).s3.amazonaws.com/', url)
    if match:
        return match.group(1), None

    match =  re.search('^https?://(.+).s3-([^.]+).amazonaws.com/', url)
    if match:
        return match.group(1), match.group(2)

    match = re.search('^https?://s3.amazonaws.com/([^\/]+)', url)
    if match:
        return match.group(1), None

    match =  re.search('^https?://s3-([^.]+).amazonaws.com/([^\/]+)', url)
    if match:
        return match.group(2), match.group(1)

    return None, None

像这样的东西应该真的进入博托......亚马逊,我希望你在听

编辑10/10/2018:存储桶正则表达式现在应该捕获具有句点的存储桶名称.