boto3 put_object with new key python

Question

I want to save a csv file ("test.csv") in S3 using boto3. my bucket is "outputS3Bucket" and the key is "folder/newFolder". I want to check if "newFolder" exists and if not to create it.

import boto3
client = boto3.client('s3')
s3 = boto3.resource('s3')
bucket = s3.Bucket("outputS3Bucket")

result = client.list_objects(Bucket='outputS3Bucket',Prefix="folder/newFolder")

if len(result)==0:
    key = bucket.new_key("folder/newFolder")
    newKey = key + "/" + "test.csv"

client.put_object(Bucket="outputS3Bucket", Key=newKey, Body=content)
# put_object path: 's3://outputS3Bucket/folder/newFolder/test.csv'

I have few problems:

1. if I don't write the full key name (such as "folder/ne") and there is a "neaFo" folder instead it still says it exists.

2. key = bucket.new_key("folder/newFolder") AttributeError: 's3.Bucket' object has no attribute 'new_key'

Answer 1

首先，根据 boto3 文档，最好使用新的 API 方法 -list_objects_v2()而不是列出存储桶的对象。

我建议使用一个简单的布尔函数来检查文件夹是否存在（使您的代码更清晰且更具可读性）。对于问题 1，您可以检查前缀是否以 '/' 字符结尾，如果不是则附加它， - 这将确保您正在寻找完全匹配而不是Starts With。

示例功能：

def bucket_folder_exists(client, bucket, path_prefix):
    # make path_prefix exact match and not path/to/folder*
    if list(path_prefix)[-1] is not '/':
        path_prefix += '/'

    # check if 'Contents' key exist in response dict - if it exist it indicate the folder exists, otherwise response will be None
    response = client.list_objects_v2(Bucket=bucket, Prefix=path_prefix).get('Contents')

    if response:
        return True
    return False

示例实施：

if bucket_folder_exists(client, 'outputS3Bucket', 'folder/newFolder'):
    pass # Do something if folder already exist
else:
    pass # Do something if folder does not exist

关于你的第二个问题，我添加了一条评论 - 似乎你的代码提到了存储桶变量\对象用作key = bucket.new_key("folder/newFolder")，但是bucket没有在代码中的任何位置设置， - >根据你收到的错误，它看起来像一个s3.Bucket对象，但它并不没有new_key定义属性。