如何在Spark中打印特定RDD分区的元素？

Question

如何单独打印特定分区的元素,比如说第5个？

val distData = sc.parallelize(1 to 50, 10)

Answer 1

使用Spark/Scala:

val data = 1 to 50
val distData = sc.parallelize(data,10)
distData.mapPartitionsWithIndex( (index: Int, it: Iterator[Int]) =>it.toList.map(x => if (index ==5) {println(x)}).iterator).collect

生产:

26
27
28
29
30