Azure Blob - 使用Python读取

Question

有人能告诉我是否可以直接从Azure blob存储中读取csv文件作为流并使用Python处理它？我知道它可以使用C#.Net(如下所示)完成,但想知道Python中的等效库来执行此操作.

CloudBlobClient client = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = client.GetContainerReference("outfiles");
CloudBlob blob = container.GetBlobReference("Test.csv");*

Answer 1

是的,当然可以这样做.查看Azure Storage SDK for Python

from azure.storage.blob import BlockBlobService

block_blob_service = BlockBlobService(account_name='myaccount', account_key='mykey')

block_blob_service.get_blob_to_path('mycontainer', 'myblockblob', 'out-sunset.png')

您可以在此处阅读完整的SDK文档:http://azure-storage.readthedocs.io.

Answer 2

以下是使用新版 SDK (12.0.0)执行此操作的方法：

from azure.storage.blob import BlobClient

blob = BlobClient(account_url="https://<account_name>.blob.core.windows.net"
                  container_name="<container_name>",
                  blob_name="<blob_name>",
                  credential="<account_key>")

with open("example.csv", "wb") as f:
    data = blob.download_blob()
    data.readinto(f)

有关详细信息，请参见此处。

Answer 3

可以使用 python 从 blob 进行流式传输，如下所示：

from tempfile import NamedTemporaryFile
from azure.storage.blob.blockblobservice import BlockBlobService

entry_path = conf['entry_path']
container_name = conf['container_name']
blob_service = BlockBlobService(
            account_name=conf['account_name'],
            account_key=conf['account_key'])

def get_file(filename):
    local_file = NamedTemporaryFile()
    blob_service.get_blob_to_stream(container_name, filename, stream=local_file, 
    max_connections=2)

    local_file.seek(0)
    return local_file

Answer 4

我建议使用smart_open。

import os

from azure.storage.blob import BlobServiceClient
from smart_open import open

connect_str = os.environ['AZURE_STORAGE_CONNECTION_STRING']
transport_params = {
    'client': BlobServiceClient.from_connection_string(connect_str),
}

# stream from Azure Blob Storage
with open('azure://my_container/my_file.txt', transport_params=transport_params) as fin:
    for line in fin:
        print(line)

# stream content *into* Azure Blob Storage (write mode):
with open('azure://my_container/my_file.txt', 'wb', transport_params=transport_params) as fout:
    fout.write(b'hello world')