小编zap*_*tar的帖子

如何从嵌套的struct元素数组创建Spark DataFrame?

我在Spark中读过一个JSON文件.该文件具有以下结构:

scala> tweetBlob.printSchema
root
 |-- related: struct (nullable = true)
 |    |-- next: struct (nullable = true)
 |    |    |-- href: string (nullable = true)
 |-- search: struct (nullable = true)
 |    |-- current: long (nullable = true)
 |    |-- results: long (nullable = true)
 |-- tweets: array (nullable = true)
 |    |-- element: struct (containsNull = true)
 |    |    |-- cde: struct (nullable = true)
...
...
 |    |    |-- cdeInternal: struct (nullable = true)
...
...
 |    | …
Run Code Online (Sandbox Code Playgroud)

scala dataframe apache-spark apache-spark-sql

8
推荐指数
3
解决办法
3万
查看次数

标签 统计

apache-spark ×1

apache-spark-sql ×1

dataframe ×1

scala ×1