使用 s3fs 下载文件

Question

我正在尝试使用 s3fs 库从 s3 存储桶下载 csv 文件。我注意到使用 Pandas 编写新的 csv 以某种方式改变了数据。所以我想直接下载原始状态的文件。

该文档具有下载功能，但我不明白如何使用它：

download(self, rpath, lpath[, recursive])： Alias of FilesystemSpec.get.

这是我尝试过的：

import pandas as pd
import datetime
import os
import s3fs
import numpy as np

#Creds for s3
fs = s3fs.S3FileSystem(key=mykey, secret=mysecretkey)
bucket = "s3://mys3bucket/mys3bucket"
files = fs.ls(bucket)[-3:]


#download files:
for file in files:
    with fs.open(file) as f:
        fs.download(f,"test.csv")

AttributeError: 'S3File' object has no attribute 'rstrip'

Answer 1

for file in files:
    fs.download(file,'test.csv')

修改为下载目录下的所有文件：

import pandas as pd
import datetime
import os
import s3fs
import numpy as np

#Creds for s3
fs = s3fs.S3FileSystem(key=mykey, secret=mysecretkey)
bucket = "s3://mys3bucket/mys3bucket"

#files references the entire bucket.
files = fs.ls(bucket)

for file in files:
    fs.download(file,'test.csv')