无法获取 Amazon S3 文件的内容并使用 python 和 boto3 编辑该文件

Question

我正在尝试从 Amazon S3 中的文件获取数据，操作内容，然后将其保存到另一个存储桶。

import json
import urllib.parse
import boto3

print('Loading function')


s3 = boto3.client('s3')

def lambda_handler(event, context):
    
    bucket = event['Records'][0]['s3']['bucket']['name']
    file_name = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'], encoding='utf-8')
    s3_object = s3.get_object(Bucket=bucket, Key=file_name)
    file_content = s3_object['Body'].read()
    
    initial_data = json.load(file_content)
    # some file manipulation comes here
    
    
    data=json.dumps(initial_data, ensure_ascii=False)
    s3.put_object(Bucket="new bucket name", Body=data, Key=file_name)        

错误消息让我认为这与编码有关：

回复：

{
  "errorMessage": "'bytes' object has no attribute 'read'",
  "errorType": "AttributeError",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 25, in lambda_handler\n    data_initlal = json.load(file_content)\n",
    "  File \"/var/lang/lib/python3.8/json/__init__.py\", line 293, in load\n    return loads(fp.read(),\n"
  ]
}

此外，如果我从代码中删除以下行：

initial_data = json.load(file_content)

我收到错误：

Response:
{
  "errorMessage": "Object of type bytes is not JSON serializable",
  "errorType": "TypeError",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 29, in lambda_handler\n    data=json.dumps(file_content, ensure_ascii=False)\n",
    "  File \"/var/lang/lib/python3.8/json/__init__.py\", line 234, in dumps\n    return cls(\n",
    "  File \"/var/lang/lib/python3.8/json/encoder.py\", line 199, in encode\n    chunks = self.iterencode(o, _one_shot=True)\n",
    "  File \"/var/lang/lib/python3.8/json/encoder.py\", line 257, in iterencode\n    return _iterencode(o, 0)\n",
    "  File \"/var/lang/lib/python3.8/json/encoder.py\", line 179, in default\n    raise TypeError(f'Object of type {o.__class__.__name__} '\n"
  ]
}

我尝试编辑的文件是 json 格式，输出也应该是 json。

Answer 1

这行：

initial_data = json.load(file_content)

应该：

initial_data = json.loads(file_content)

或者，替换这两行：

initial_data = json.load(file_content)

和：

initial_data = json.loads(file_content)

区别在于json.load()vs json.loads()。