在 Athena 中分解数组

Question

我在 athena 中有一个简单的表，它有一系列事件。我想编写一个简单的选择语句，以便数组中的每个事件成为一行。

我尝试过爆炸、变形，但没有成功。我已经在 Spark 和 Hive 中成功完成了。但这雅典娜却在欺骗我。请指教

DROP TABLE bi_data_lake.royalty_v4;
CREATE external TABLE bi_data_lake.royalty_v4 (
   KAFKA_ID string,
   KAFKA_TS string,
   deviceUser struct< deviceName:string, devicePlatform:string >,
   consumeReportingEvents array<
                                struct<
                                        consumeEvent: string,
                                        consumeEventAction: string,
                                        entryDateTime: string
                                      >
                               >
   )
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 's3://XXXXXXXXXXX';

查询哪个不起作用

select kafka_id, kafka_ts,deviceuser, 
transform( consumereportingevents, consumereportingevent -> consumereportingevent.consumeevent) as cre
from bi_data_lake.royalty_v4 
where kafka_id = 'events-consumption-0-490565';

不支持横向视图分解（consumereportingevents）作为consumereportingevent

回答问题使用unnset

找到了我的问题的答案

WITH samples AS (
 select kafka_id, kafka_ts,deviceuser, consumereportingevent, consumereportingeventPos
 from bi_data_lake.royalty_v4 
 cross join unnest(consumereportingevents)  WITH ORDINALITY AS T (consumereportingevent, consumereportingeventPos)
 where kafka_id = 'events-consumption-0-490565' or kafka_id = 'events-consumption-0-490566'
)
SELECT * FROM samples

Answer 1

使用 .AWS Athena 压平（“爆炸”）嵌套UNNEST数组

WITH dataset AS (
  SELECT
    'engineering' as department,
    ARRAY['Sharon', 'John', 'Bob', 'Sally'] as users
)
SELECT department, names FROM dataset
CROSS JOIN UNNEST(users) as t(names)

参考：展平嵌套数组