Python-从Google Cloud Storage下载整个目录

Question

在下一页

https://googlecloudplatform.github.io/google-cloud-python/latest/storage/blobs.html

有所有可用于Python和Google Cloud存储的API调用。即使在github上的“官方”样本中

https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/storage/cloud-client/snippets.py

没有相关的例子。

最后，使用与下载文件相同的方法下载目录会出现错误

Error:  [Errno 21] Is a directory:

Answer 1

如果您想保持相同的目录结构而不重命名并创建嵌套文件夹。对于 python 3.5+，我有一个基于 @ksbg 答案的解决方案：

from pathlib import Path
bucket_name = 'your-bucket-name'
prefix = 'your-bucket-directory/'
dl_dir = 'your-local-directory/'

storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket_name=bucket_name)
blobs = bucket.list_blobs(prefix=prefix)  # Get list of files
for blob in blobs:
    if blob.name.endswith("/"):
        continue
    file_split = blob.name.split("/")
    directory = "/".join(file_split[0:-1])
    Path(directory).mkdir(parents=True, exist_ok=True)
    blob.download_to_filename(blob.name) 

Answer 2

您只需要首先列出目录中的所有文件，然后一个个下载它们：

bucket_name = 'your-bucket-name'
prefix = 'your-bucket-directory/'
dl_dir = 'your-local-directory/'

storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket_name=bucket_name)
blobs = bucket.list_blobs(prefix=prefix)  # Get list of files
for blob in blobs:
    filename = blob.name.replace('/', '_') 
    blob.download_to_filename(dl_dir + filename)  # Download

blob.name包括整个目录结构+文件名，所以如果你想相同的文件名如斗，你可能需要先解包（而不是替换/用_）