小编wil*_*eng的帖子

我第一次尝试从 Google 云存储下载文件。

我设置了从https://cloud.google.com/storage/docs/reference/libraries#client-libraries-usage-python下载的 googstruct.json 服务帐户密钥文件的路径

是否需要在代码之外设置对 Google Cloud 的授权？或者是否有比谷歌网站上更好的“如何使用谷歌云存储”？
看来我将错误的类型传递给storage_client = storage.Client() 异常字符串如下。

发生异常：google.auth.exceptions.DefaultCredentialsError 文件 C:\Users\Cary\Documents\Programming\Python\QGIS\GoogleCloud\googstruct.json 没有有效类型。类型为 None，应为 ('authorized_user', 'service_account') 之一。

我的 Python 3.7 代码

from google.cloud import storage
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="C:\\GoogleCloud\\googstruct.json"

# Instantiates a client
storage_client = storage.Client()
bucket_name = 'structure_ssi'
destination_file_name = "C:\\Users\\18809_PIPEM.shp"
source_blob_name = '18809_PIPEM.shp'
download_blob(bucket_name, source_blob_name, destination_file_name)

def download_blob(bucket_name, source_blob_name, destination_file_name):
    """Downloads a blob from the bucket."""
    storage_client = storage.Client()
    bucket = storage_client.get_bucket(bucket_name)
    blob = bucket.blob(source_blob_name)

    blob.download_to_filename(destination_file_name)

    print('Blob {} downloaded to {}.'.format(
        source_blob_name, …

我得到了以下格式的具有嵌套结构的 JSON 文档

{
    "id": "p-1234-2132321-213213213-12312",
    "name": "athena to the rescue",
    "groups": [
        {
            "strategy_group": "anyOf",
            "conditions": [
                {
                    "strategy_conditions": "anyOf",
                    "entries": [
                        {
                            "c_key": "service",
                            "C_operation": "isOneOf",
                            "C_value": "mambo,bambo,jumbo"
                        },
                        {
                            "c_key": "hostname",
                            "C_operation": "is",
                            "C_value": "lols"
                        }
                    ]
                }
            ]
        }
    ],
    "tags": [
        "aaa",
        "bbb",
        "ccc"
    ]
}

我在 Athena 中创建了表来支持它，使用以下内容

CREATE EXTERNAL TABLE IF NOT EXISTS filters ( id string, name string, tags array<string>, groups array<struct<
    strategy_group:string,
    conditions:array<struct<
        strategy_conditions:string,
        entries: array<struct<
            c_key:string,
            c_operation:string,
            c_value:string
        >>
    >> …

我不确定如何解决这个问题——其他帖子ModuleNotFoundError是通过重新安装相关包来解决的，但是很明显这不是问题，因为示例 bash 运算符 DAG 运行。那么我的问题与 Airflow 的安装方式有关吗？此时，我正在考虑通过puckel Docker 容器重新安装 Airflow 。

[2020-12-23 01:02:24,705] {dagbag.py:259} ERROR - Failed to import: /home/pi/airflow/dags/dag_clist.py

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/airflow/models/dagbag.py", line 256, in process_file
    m = imp.load_source(mod_name, filepath)
  File "/usr/lib/python3.7/imp.py", line 171, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 696, in _load
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/pi/airflow/dags/dag_clist.py", line 7, …