我将以下JSON作为DataFrame加载:
root
|-- data: struct (nullable = true)
| |-- field1: string (nullable = true)
| |-- field2: string (nullable = true)
|-- moreData: array (nullable = true)
| |-- element: struct (containsNull = true)
| | |-- more1: string (nullable = true)
| | |-- more2: string (nullable = true)
| | |-- more3: string (nullable = true)
我想从这个DataFrame中获取以下RDD:
RDD[(more1, more2, more3, field1, field2)]
我怎样才能做到这一点？我想我必须使用flatMap嵌套的JSON？