我尝试在 AWS S3 SELECT Web UI 中选择 CSV,但收到以下错误消息:
Quoted record delimiter found in the file. To allow quoted record delimiters, please set AllowQuotedRecordDelimiter to 'TRUE'.
Run Code Online (Sandbox Code Playgroud)
但没有选项可以AllowQuotedRecordDelimiter设置TRUE
请指教
我正在尝试使用以下代码从存储在 S# 存储桶中的 CSV 中获取记录的子集:
s3 = boto3.client('s3')
bucket = bucket
file_name = file
sql_stmt = """SELECT S.* FROM s3object S LIMIT 10"""
req = s3.select_object_content(
Bucket=bucket,
Key=file,
ExpressionType='SQL',
Expression=sql_stmt,
InputSerialization = {'CSV': {'FileHeaderInfo': 'USE'}},
OutputSerialization = {'CSV': {}},
)
records = []
for event in req['Payload']:
if 'Records' in event:
records.append(event['Records']['Payload'])
elif 'Stats' in event:
stats = event['Stats']['Details']
file_str = ''.join(r.decode('utf-8') for r in records)
select_df = pd.read_csv(StringIO(file_str))
df = pd.DataFrame(select_df)
print(df)
Run Code Online (Sandbox Code Playgroud)
这成功地产生了记录但错过了标题。
我在这里读到S3 Select CSV Headers,S3 …