使用 AWS Lambda (python) 编写 csv 文件并将其保存到 S3 中

Question

我正在尝试使用 AWS Lambda 将 csv 文件写入 S3 存储桶，为此我使用了以下代码：

data=[[1,2,3],[23,56,98]]
with open("s3://my_bucket/my_file.csv", "w") as f:
   f.write(data)

这会引发以下错误：

[Errno 2] No such file or directory: u's3://my_bucket/my_file.csv': IOError
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 51, in lambda_handler
with open("s3://my_bucket/my_file.csv", "w") as f:
IOError: [Errno 2] No such file or directory: u's3://my_bucket/my_file.csv'

请问我能帮上忙吗？

PS：我使用的是 python 2.7

提前致谢

Answer 1

迟到总比不答复好。在 S3 中获取数据有四个步骤：

• 调用 S3 存储桶
• 使用请求库将数据加载到 Lambda 中（如果你没有安装它，你将不得不将它作为一个层加载）
• 将数据写入 Lambda '/tmp' 文件
• 将文件上传到s3

像这样的东西：

import csv
import requests
#all other apropriate libs already be loaded in lambda

#properly call your s3 bucket
s3 = boto3.resource('s3')
bucket = s3.Bucket('your-bucket-name')
key = 'yourfilename.txt'

#you would need to grab the file from somewhere. Use this incomplete line below to get started:
with requests.Session() as s:
    getfile = s.get('yourfilelocation')

#Only then you can write the data into the '/tmp' folder.
with open('/tmp/yourfilename.txt', 'w', newline='') as f:
    w = csv.writer(f)
    w.writerows(filelist)
#upload the data into s3
bucket.upload_file('/tmp/yourfilename.txt', key)

希望能帮助到你。

Answer 2

with open("s3://my_bucket/my_file.csv", "w+") as f:

代替

with open("s3://my_bucket/my_file.csv", "w") as f:

请注意，“w”已更改为“w+”，这意味着它将写入该文件，如果该文件不存在，它将创建它。