Bigquery python SchemaField() 与结构数组

Question

我正在尝试通过 python 客户端在 Bigquery 创建一个表。该文档用于bigquery.SchemaField('name', 'TYPE')定义一个字段。尽管它似乎不适用于ARRAYS或STRUCTS。这是我试图创建的 STRUCTS 字段的数组：

bigquery.SchemaField('owners', 'ARRAY<STRUCT<emailAddress STRING, displayName STRING>>', 'REPEATABLE'),

如果我使用上面的字段定义，我会收到以下 API 错误：

400 POST https://www.googleapis.com/bigquery/v2/projects/import-sheet/datasets/sheetgo/tables: Invalid value for: ARRAY<STRUCT<emailAddress STRING, displayName STRING>> is not a valid value

整个代码：

schema = [
            bigquery.SchemaField('user', 'STRING'),
            bigquery.SchemaField('id', 'STRING'),
            bigquery.SchemaField('service_origin', 'STRING'),
            bigquery.SchemaField('name', 'STRING'),
            bigquery.SchemaField('mimeType', 'STRING'),
            bigquery.SchemaField('createdAt', 'DATETIME'),
            bigquery.SchemaField('ownedByMe', 'BOOLEAN'),
            bigquery.SchemaField('owners', 'ARRAY<STRUCT<emailAddress STRING, displayName STRING>>', 'REPEATABLE'),
            bigquery.SchemaField('parents', 'ARRAY<STRING>', 'REPEATABLE'),
            bigquery.SchemaField('teamDriveId', 'STRING'),
            bigquery.SchemaField('permissions', 'STRING'),
            bigquery.SchemaField('shared', 'BOOLEAN'),
            bigquery.SchemaField('writersCanShare', 'BOOLEAN'),
            bigquery.SchemaField('sharingUser', 'STRING'),
            bigquery.SchemaField('version', 'STRING'),
            bigquery.SchemaField('size', 'FLOAT'),
            bigquery.SchemaField('data_properties', 'ARRAY<STRUCT<'
                                                        'rows INTEGER,'
                                                        'cells_with_importrange ARRAY<'
                                                                                    'STRUCT<'
                                                                                        'row_index INTEGER,'
                                                                                        'col_index INTEGER,'
                                                                                        'importrange STRING'
                                                                                      '>'
                                                                                    '>,'
                                                        'tab_name STRING,'
                                                        'cell_count FLOAT,'
                                                        'header_rows ARRAY<STRING>,'
                                                    '>>', 'REPEATABLE'),
            bigquery.SchemaField('timezone', 'STRING'),
            bigquery.SchemaField('locale', 'STRING'),
            bigquery.SchemaField('last_scansheet', 'STRING'),
        ]

        bigquery_client = bigquery.Client(PROJECT_ID)
        dataset_ref = bigquery_client.dataset("eita")

        table_ref = dataset_ref.table(table_id)
        table = bigquery.Table(table_ref, schema=schema)

        table = bigquery_client.create_table(table)

更新

感谢Willian Fuks，我成功了。模式的最终结果如下：

schema = [
            bigquery.SchemaField('user', 'STRING'),
            bigquery.SchemaField('id', 'STRING'),
            bigquery.SchemaField('service_origin', 'STRING'),
            bigquery.SchemaField('name', 'STRING'),
            bigquery.SchemaField('mimeType', 'STRING'),
            bigquery.SchemaField('createdAt', 'DATETIME'),
            bigquery.SchemaField('ownedByMe', 'BOOLEAN'),
            bigquery.SchemaField('owners', 'RECORD', mode='REPEATED',
                fields=(
                    bigquery.SchemaField('emailAddress', 'STRING'),
                    bigquery.SchemaField('displayName', 'STRING')
                )
            ),
            bigquery.SchemaField('parents', 'STRING', mode='REPEATED'),
            bigquery.SchemaField('teamDriveId', 'STRING'),
            bigquery.SchemaField('permissions', 'STRING'),
            bigquery.SchemaField('shared', 'BOOLEAN'),
            bigquery.SchemaField('writersCanShare', 'BOOLEAN'),
            bigquery.SchemaField('sharingUser', 'STRING'),
            bigquery.SchemaField('version', 'STRING'),
            bigquery.SchemaField('size', 'FLOAT'),
            bigquery.SchemaField('data_properties', 'RECORD', mode='REPEATED',
                 fields=(
                     bigquery.SchemaField('rows', 'INTEGER'),
                     bigquery.SchemaField('cells_with_importrange', 'RECORD', mode='REPEATED',
                          fields=(
                              bigquery.SchemaField('row_index', 'INTEGER'),
                              bigquery.SchemaField('col_index', 'INTEGER'),
                              bigquery.SchemaField('importrange', 'STRING'),
                          )
                      ),
                     bigquery.SchemaField('tab_name', 'STRING'),
                     bigquery.SchemaField('cell_count', 'FLOAT'),
                     bigquery.SchemaField('header_rows', 'STRING', mode='REPEATED')
                 )
             ),
            bigquery.SchemaField('timezone', 'STRING'),
            bigquery.SchemaField('locale', 'STRING'),
            bigquery.SchemaField('last_scansheet', 'STRING'),
        ]

Answer 1

构造函数的合约SchemaField确实期望与您使用的输入不同的输入。

试试这个：

schema = [
    (...),
    SchemaField('owners', 'RECORD', mode='REPEATED',
        fields=(SchemaField('emailAddress', 'STRING'),
                SchemaField('displayName', 'STRING')
        )
    ),
    (...)
]

主要思想是通过使用其他定义来定义记录字段内的字段SchemaField。