小编Sop*_*unk的帖子

将 Pandas Dataframe 写入_csv StringIO 而不是文件

客观此代码的是读取从指定S3桶现有CSV文件到一个数据帧,过滤数据帧为所需的列,然后编写过滤数据帧使用StringIO的一个CSV对象,我可以上传到不同S3桶。

除了函数“prepare_file_for_upload”的代码块之外,现在一切正常。下面是完整的代码块:

from io import StringIO
import io #unsued at the moment
import logging
import pandas as pd
import boto3
from botocore.exceptions import ClientError

FORMAT = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
logging.basicConfig(level=logging.INFO, format=FORMAT)
logger = logging.getLogger(__name__)

#S3 parameters
source_bucket = 'REPLACE'
source_folder = 'REPLACE/'
dest_bucket = 'REPLACE'
dest_folder = 'REPLACE'
output_name = 'REPLACE'

def get_file_name():
try:
    s3 = boto3.client("s3")
    logging.info(f'Determining filename from: {source_bucket}/{source_folder}')
    bucket_path = s3.list_objects(Bucket=source_bucket, Prefix=source_folder)
    file_name =[key['Key'] for key in bucket_path['Contents']][1] …
Run Code Online (Sandbox Code Playgroud)

dataframe python-3.x export-to-csv pandas boto3

5
推荐指数
2
解决办法
6932
查看次数

标签 统计

boto3 ×1

dataframe ×1

export-to-csv ×1

pandas ×1

python-3.x ×1