从 Databricks 挂载 ADLS Gen 2 abfss 时出错：错误 IllegalArgumentException：不支持的 Azure 方案：abfss

Question

我修改了 @Bhagyashree 善意提供的一些代码，试图在 ADLS Gen 2 上安装 abfss

container_name = "root"
storage_account = "mystorageaccount"
key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

url = "abfss://" + container_name + "@" + storage_account + ".dfs.core.windows.net/"
config = "fs.azure.account.key." + storage_account + ".blob.core.windows.net"

mount_folder = "/mnt/lake"
mounted_list = dbutils.fs.mounts()

mounted_exist = False
for item in mounted_list:
  if mount_folder in item[0]:
    mounted_exist = True
    break

  if not mounted_exist:
    dbutils.fs.mount(source = url, mount_point = mount_folder, extra_configs = {config : key})

目标是实现如下所示的安装

但我得到了错误

IllegalArgumentException: Unsupported Azure Scheme: abfss

关于为什么我收到错误有什么想法吗？

Answer 1

• 当我尝试使用帐户密钥安装 azure data Lake 存储的相同代码时，出现了相同的错误。

• 安装abfss可能需要OAuth身份验证，因此使用帐户密钥安装 abfss 会引发错误。修改代码如下所示，我能够挂载并列出其内容：

container_name = "data"
storage_account = "datalk230523"
key = "<account_key>"

url = "wasbs://" + container_name + "@" + storage_account + ".blob.core.windows.net/"
config = "fs.azure.account.key." + storage_account + ".blob.core.windows.net"

mount_folder = "/mnt/repro"
dbutils.fs.mount(source = url, mount_point = mount_folder, extra_configs = {config : key})

• 使用dbutils.fs.ls，我能够成功列出内容。以下是输出图像供参考：

注意：挂载后需要修改循环逻辑，因为也是第二次迭代，挂载列表中没有最新的挂载信息。